jacobgil / pytorch-pruning Goto Github PK

Could it was pruned to ResNet model?
If ResNet model loaded, it occured error to FilterPrunner.

when running the project, errors occurs and don't know why

maybe someone can help me with this?

I run python finetune.py --train
the output accuracy is always 0
I use the dataset Dogs vs. Cats

Hi,

I download kaggle train and test datasets, but it seems that the labels in test are missed.

Any advice?

Hi Jacob! Thanks for your impressive work. I ran your code pruning VGG16 by removing 60% filters. However, the runtime for testing only speeds up from 0.64sec/image to 0.39sec/image, which is about 1.6x faster. I'm not sure if the way I monitor the runtime is correct. My current approach is setting the starting point before the forward pass in the test function (above the line 144) and setting the end point after the prediction (below the line 145). Also, I'm currently using CPU for training and testing; so I'm not sure if it has some adverse effect with respect to GPU.

Thanks!

When I use my own modified pytorch-pruning for python 3 with this fruits dataset , I have quite low accuracy value. May I know why ?

Besides, how do we use the output model_prunned ? Do we have to write our own pytorch code to test any random input images not in the test and training set ?

[phung@archlinux pytorch-pruning]$ ls -al test/
total 16
drwxr-xr-x 2 phung phung 4096 Oct 20 20:55 .
drwxr-xr-x 6 phung phung 4096 Oct 20 23:57 ..
lrwxrwxrwx 1 phung phung 96 Oct 20 20:54 Lemon -> /home/phung/Documents/Grive/Personal/Coursera/Machine_Learning/dataset/kaggle_fruits/Test/Lemon/
lrwxrwxrwx 1 phung phung 97 Oct 20 20:55 Orange -> /home/phung/Documents/Grive/Personal/Coursera/Machine_Learning/dataset/kaggle_fruits/Test/Orange/
[phung@archlinux pytorch-pruning]$ ls -al train/
total 16
drwxr-xr-x 2 phung phung 4096 Oct 20 20:54 .
drwxr-xr-x 6 phung phung 4096 Oct 20 23:57 ..
lrwxrwxrwx 1 phung phung 100 Oct 20 20:54 Lemon -> /home/phung/Documents/Grive/Personal/Coursera/Machine_Learning/dataset/kaggle_fruits/Training/Lemon/
lrwxrwxrwx 1 phung phung 100 Oct 20 20:53 Orange -> /home/phung/Documents/Grive/Personal/Coursera/Machine_Learning/dataset/kaggle_fruits/Training/Lemon/
[phung@archlinux pytorch-pruning]$ python finetune.py --train && python finetune.py --prune
/usr/lib/python3.7/site-packages/torchvision/transforms/transforms.py:187: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
warnings.warn("The use of the transforms.Scale transform is deprecated, " +
/usr/lib/python3.7/site-packages/torchvision/transforms/transforms.py:562: UserWarning: The use of the transforms.RandomSizedCrop transform is deprecated, please use transforms.RandomResizedCrop instead.
warnings.warn("The use of the transforms.RandomSizedCrop transform is deprecated, " +
Epoch: 0
Accuracy: 0.49382716049382713
Epoch: 1
Accuracy: 0.49382716049382713
Epoch: 2
Accuracy: 0.4876543209876543
Epoch: 3
Accuracy: 0.5339506172839507
Epoch: 4
Accuracy: 0.49382716049382713
Epoch: 5
Accuracy: 0.49382716049382713
Epoch: 6
Accuracy: 0.49382716049382713
Epoch: 7
Accuracy: 0.3765432098765432
Epoch: 8
Accuracy: 0.49382716049382713
Epoch: 9
Accuracy: 0.5
Epoch: 10
Accuracy: 0.49382716049382713
Epoch: 11
Accuracy: 0.49382716049382713
Epoch: 12
Accuracy: 0.49382716049382713
Epoch: 13
Accuracy: 0.24691358024691357
Epoch: 14
Accuracy: 0.3333333333333333
Epoch: 15
Accuracy: 0.4845679012345679
Epoch: 16
Accuracy: 0.5
Epoch: 17
Accuracy: 0.49382716049382713
Epoch: 18
Accuracy: 0.5030864197530864
Epoch: 19
Accuracy: 0.49382716049382713
Finished fine tuning.
/usr/lib/python3.7/site-packages/torchvision/transforms/transforms.py:187: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
warnings.warn("The use of the transforms.Scale transform is deprecated, " +
/usr/lib/python3.7/site-packages/torchvision/transforms/transforms.py:562: UserWarning: The use of the transforms.RandomSizedCrop transform is deprecated, please use transforms.RandomResizedCrop instead.
warnings.warn("The use of the transforms.RandomSizedCrop transform is deprecated, " +
Accuracy: 0.49382716049382713
Number of prunning iterations to reduce 67% filters 5
Ranking filters..
Layers that will be prunned {17: 62, 28: 117, 10: 19, 24: 57, 12: 20, 7: 8, 19: 57, 0: 9, 26: 60, 21: 72, 14: 22, 5: 6, 2: 3}
Prunning filters..
Filters prunned 87.87878787878788%
Accuracy: 0.49382716049382713
Fine tuning to recover from prunning iteration.
Epoch: 0
Accuracy: 0.5061728395061729
Epoch: 1
Accuracy: 0.4722222222222222
Epoch: 2
Accuracy: 0.5061728395061729
Epoch: 3
Accuracy: 0.49382716049382713
Epoch: 4
Accuracy: 0.558641975308642
Epoch: 5
Accuracy: 0.7469135802469136
Epoch: 6
Accuracy: 0.7314814814814815
Epoch: 7
Accuracy: 0.6234567901234568
Epoch: 8
Accuracy: 0.5061728395061729
Epoch: 9
Accuracy: 0.5401234567901234
Finished fine tuning.
Ranking filters..
Layers that will be prunned {28: 97, 21: 88, 19: 53, 17: 42, 24: 63, 26: 103, 12: 16, 5: 5, 10: 18, 14: 18, 7: 6, 2: 2, 0: 1}
Prunning filters..
Filters prunned 75.75757575757575%
Accuracy: 0.49382716049382713
Fine tuning to recover from prunning iteration.
Epoch: 0
Accuracy: 0.5061728395061729
Epoch: 1
Accuracy: 0.404320987654321
Epoch: 2
Accuracy: 0.24691358024691357
Epoch: 3
Accuracy: 0.5030864197530864
Epoch: 4
Accuracy: 0.4012345679012346
Epoch: 5
Accuracy: 0.5061728395061729
Epoch: 6
Accuracy: 0.5061728395061729
Epoch: 7
Accuracy: 0.5061728395061729
Epoch: 8
Accuracy: 0.5061728395061729
Epoch: 9
Accuracy: 0.5061728395061729
Finished fine tuning.
Ranking filters..
Layers that will be prunned {28: 90, 26: 64, 12: 22, 24: 87, 17: 62, 21: 52, 7: 8, 5: 8, 19: 63, 10: 24, 0: 4, 14: 26, 2: 2}
Prunning filters..
Filters prunned 63.63636363636363%
Accuracy: 0.5061728395061729
Fine tuning to recover from prunning iteration.
Epoch: 0
Accuracy: 0.5061728395061729
Epoch: 1
Accuracy: 0.49382716049382713
Epoch: 2
Accuracy: 0.49382716049382713
Epoch: 3
Accuracy: 0.6604938271604939
Epoch: 4
Accuracy: 1.0
Epoch: 5
Accuracy: 0.6790123456790124
Epoch: 6
Accuracy: 0.5061728395061729
Epoch: 7
Accuracy: 0.5061728395061729
Epoch: 8
Accuracy: 0.5679012345679012
Epoch: 9
Accuracy: 0.9845679012345679
Finished fine tuning.
Ranking filters..
Layers that will be prunned {28: 75, 14: 25, 26: 74, 21: 88, 19: 61, 7: 14, 17: 49, 24: 66, 10: 25, 0: 6, 12: 18, 5: 7, 2: 4}
Prunning filters..
Filters prunned 51.515151515151516%
Accuracy: 0.6358024691358025
Fine tuning to recover from prunning iteration.
Epoch: 0
Accuracy: 0.7623456790123457
Epoch: 1
Accuracy: 0.7623456790123457
Epoch: 2
Accuracy: 0.6851851851851852
Epoch: 3
Accuracy: 0.8950617283950617
Epoch: 4
Accuracy: 0.9660493827160493
Epoch: 5
Accuracy: 1.0
Epoch: 6
Accuracy: 0.49382716049382713
Epoch: 7
Accuracy: 0.49691358024691357
Epoch: 8
Accuracy: 0.9691358024691358
Epoch: 9
Accuracy: 0.9783950617283951
Finished fine tuning.
Ranking filters..
Layers that will be prunned {28: 73, 26: 70, 24: 82, 19: 78, 14: 35, 17: 56, 21: 49, 0: 3, 5: 11, 12: 27, 10: 19, 7: 6, 2: 3}
Prunning filters..
Filters prunned 39.39393939393939%
Accuracy: 0.9074074074074074
Fine tuning to recover from prunning iteration.
Epoch: 0
Accuracy: 0.8364197530864198
Epoch: 1
Accuracy: 0.9691358024691358
Epoch: 2
Accuracy: 0.49382716049382713
Epoch: 3
Accuracy: 0.5092592592592593
Epoch: 4
Accuracy: 0.5864197530864198
Epoch: 5
Accuracy: 0.49382716049382713
Epoch: 6
Accuracy: 0.49382716049382713
Epoch: 7
Accuracy: 1.0
Epoch: 8
Accuracy: 1.0
Epoch: 9
Accuracy: 0.8703703703703703
Finished fine tuning.
Finished. Going to fine tune the model a bit more
Epoch: 0
Accuracy: 0.49382716049382713
Epoch: 1
Accuracy: 0.8333333333333334
Epoch: 2
Accuracy: 0.7561728395061729
Epoch: 3
Accuracy: 0.49382716049382713
Epoch: 4
Accuracy: 0.5370370370370371
Epoch: 5
Accuracy: 1.0
Epoch: 6
Accuracy: 1.0
Epoch: 7
Accuracy: 0.9135802469135802
Epoch: 8
Accuracy: 0.6666666666666666
Epoch: 9
Accuracy: 1.0
Epoch: 10
Accuracy: 0.49382716049382713
Epoch: 11
Accuracy: 1.0
Epoch: 12
Accuracy: 0.49382716049382713
Epoch: 13
Accuracy: 0.49382716049382713
Epoch: 14
Accuracy: 0.49382716049382713
Finished fine tuning.
[phung@archlinux pytorch-pruning]$ ls -al
total 645652
drwxr-xr-x 6 phung phung 4096 Oct 20 23:57 .
drwxr-xr-x 6 phung phung 4096 Oct 5 08:04 ..
-rw-r--r-- 1 phung phung 1695 Oct 15 23:50 dataset.py
-rw-r--r-- 1 phung phung 9323 Oct 16 09:16 finetune.py
drwxr-xr-x 8 phung phung 4096 Oct 20 21:53 .git
-rw-r--r-- 1 phung phung 50 Oct 20 20:58 .gitignore
-rw-r--r-- 1 phung phung 537103112 Oct 20 21:49 model
-rw-r--r-- 1 phung phung 123976929 Oct 20 23:57 model_prunned
-rw-r--r-- 1 phung phung 4939 Oct 20 21:15 prune.py
drwxr-xr-x 2 phung phung 4096 Oct 20 21:18 pycache
-rw-r--r-- 1 phung phung 1488 Sep 27 22:54 README.md
drwxr-xr-x 2 phung phung 4096 Oct 20 20:55 test
drwxr-xr-x 2 phung phung 4096 Oct 20 20:54 train
[phung@archlinux pytorch-pruning]$

Hi

I am working on Tesla K40, 12 GB GPU machine. I am facing this error constantly. If I calculate the required memory for VGG model with respect to the mentioned batch size in dataset.py , the required memory is far less than the available memory of GPU. What could be the reason and how to overcome this?
I am facing this after initializing the model and while calling cuda() also.

THCudaCheck FAIL file=/pytorch/torch/lib/THC/THCCachingHostAllocator.cpp line=258 error=2 : out of memory
Traceback (most recent call last):
File "finetune.py", line 272, in
fine_tuner.train(epoches = 20)
File "finetune.py", line 163, in train
self.train_epoch(optimizer)
File "finetune.py", line 182, in train_epoch
for batch, label in self.train_data_loader:
File "/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.py", line 281, in next
return self._process_next_batch(batch)
File "/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.py", line 301, in process_next_batch
raise batch.exc_type(batch.exc_msg)
RuntimeError: Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.py", line 81, in worker_manager_loop
batch = pin_memory_batch(batch)
File "/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.py", line 148, in pin_memory_batch
return [pin_memory_batch(sample) for sample in batch]
File "/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.py", line 142, in pin_memory_batch
return batch.pin_memory()
File "/usr/local/lib/python2.7/dist-packages/torch/tensor.py", line 92, in pin_memory
return type(self)().set(storage.pin_memory()).view_as(self)
File "/usr/local/lib/python2.7/dist-packages/torch/storage.py", line 87, in pin_memory
return type(self)(self.size(), allocator=allocator).copy(self)
RuntimeError: cuda runtime error (2) : out of memory at /pytorch/torch/lib/THC/THCCachingHostAllocator.cpp:258

how to obtain VGG16: flops = [3.1, 57.8,14.1, 28.9, 7:0, 14.5,14.5,3.5, 7.2,7.2, 1.8,1.8, 1.8, 1.8] according to FLOPs = 2HW(CinK2 + 1)Cout?
for example, no.1 conv FLOPs(0) = 3.1, while FLOPs = 2HW(CinK2 + 1)Cout =
2 * 224 * 224 * (3 * 3^2 + 1) * 64, so how to compute FLOPs(0) = 3.1?

Hi Jacob, I get this error when i run finetune.py --prune

Traceback (most recent call last):
File "fine_tune.py", line 271, in
fine_tuner.prune()
File "fine_tune.py", line 218, in prune
prune_targets = self.get_candidates_to_prune(num_filters_to_prune_per_iteration)
File "fine_tune.py", line 184, in get_candidates_to_prune
self.train_epoch(rank_filters = True)
File "fine_tune.py", line 179, in train_epoch
self.train_batch(optimizer, batch.cuda(), label.cuda(), rank_filters)
File "fine_tune.py", line 172, in train_batch
self.criterion(output, Variable(label)).backward()
File "/usr/local/lib/python2.7/dist-packages/torch/autograd/variable.py", line 156, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, retain_variables)
File "/usr/local/lib/python2.7/dist-packages/torch/autograd/init.py", line 98, in backward
variables, grad_variables, retain_graph)
File "fine_tune.py", line 77, in compute_rank
sum(dim=2).sum(dim=3)[0, :, 0, 0].data
File "/usr/local/lib/python2.7/dist-packages/torch/autograd/variable.py", line 476, in sum
return Sum.apply(self, dim, keepdim)
File "/usr/local/lib/python2.7/dist-packages/torch/autograd/_functions/reduce.py", line 21, in forward
return input.sum(dim)
RuntimeError: dimension out of range (expected to be in range of [-2, 1], but got 3)

I have not been able to figure out exactly what's causing the error

hi jacobgil

It is a very nice work, Just wondering if you have any ideas or suggestions for Batch normalization which contains the info from upstream conv layer ? We need to handle the scale and shift if any filters are pruned.

Impressive job, I want to compress and accelerate Resnet, Is it possible to use your code directly? Or need to modify some codes, If need modification, where should be modified? Thank you very much.

Can anybody explain that? Thank you very much.

@jacobgil I find there is no limits when using optimizer.step(). So the pruned weight will get a gradient and after stepping, it will be no longer 0 which means it cannot be regarded as pruned?

Am I right? Hope for your response!

I tried the project on Python3.6. Here is the log, the accuracy drops significantly, which is different from your blog result: The accuracy dropped from 98.7% to 97.5%.

$ python3 test_pruning.py --prune
CHECK GPU AVAILEBLE: True
/home/web_server/dlpy72/py3.6/lib/python3.6/site-packages/torchvision/transforms/transforms.py:156: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
  "please use transforms.Resize instead.")
/home/web_server/dlpy72/py3.6/lib/python3.6/site-packages/torchvision/transforms/transforms.py:397: UserWarning: The use of the transforms.RandomSizedCrop transform is deprecated, please use transforms.RandomResizedCrop instead.
  "please use transforms.RandomResizedCrop instead.")
Correct: 845, Failed: 31, Accuracy: 0.9646118721461188
Number of prunning iterations to reduce 67% filters 5
Ranking filters.. 
Layers that will be prunned {28: 130, 17: 56, 26: 71, 21: 53, 0: 5, 19: 60, 10: 20, 12: 20, 7: 9, 2: 4, 24: 62, 14: 13, 5: 9}
Prunning filters.. 
Filters prunned 87.87878787878788%
Correct: 838, Failed: 38, Accuracy: 0.95662100456621
Fine tuning to recover from prunning iteration.
Ranking filters.. 
Layers that will be prunned {28: 110, 26: 69, 14: 17, 24: 80, 21: 60, 10: 23, 17: 64, 7: 7, 19: 52, 12: 18, 5: 5, 0: 4, 2: 3}
Prunning filters.. 
Filters prunned 75.75757575757575%
Correct: 817, Failed: 59, Accuracy: 0.932648401826484
Fine tuning to recover from prunning iteration.
Ranking filters.. 
Layers that will be prunned {24: 80, 21: 47, 17: 75, 14: 22, 26: 92, 2: 4, 12: 23, 19: 64, 10: 21, 28: 67, 5: 8, 7: 8, 0: 1}
Prunning filters.. 
Filters prunned 63.63636363636363%
Correct: 754, Failed: 122, Accuracy: 0.860730593607306
Fine tuning to recover from prunning iteration.
Ranking filters.. 
Layers that will be prunned {26: 103, 19: 98, 14: 19, 17: 54, 21: 88, 24: 63, 12: 17, 10: 16, 28: 42, 7: 2, 2: 1, 0: 6, 5: 3}
Prunning filters.. 
Filters prunned 51.515151515151516%
Correct: 468, Failed: 408, Accuracy: 0.5342465753424658
Fine tuning to recover from prunning iteration.
Ranking filters.. 
Layers that will be prunned {21: 91, 17: 79, 5: 17, 14: 36, 19: 68, 10: 33, 12: 32, 26: 40, 0: 10, 24: 69, 2: 5, 28: 25, 7: 7}
Prunning filters.. 
Filters prunned 39.39393939393939%
Correct: 514, Failed: 362, Accuracy: 0.58675799086758
Fine tuning to recover from prunning iteration.
Finished. Going to fine tune the model a bit more

why set layer_index=28, filter_index=10?
thanks?

the speed of model pruned did not improve, how to your work?

when hundreds of classes, accuracy is nearly 0.0

Hi,
first of all thanks for the awesome work.

i'm trying ti utilize your work to a more complicated model (xception). in this model there are conv layers with groups parameter which is different then 1 (specifically, it is equal to in_channels and output_channels). if i try to prune 1 filter out of such layer, i get an error since groups needs to be divisible by both in_channels and out_channels.

any idea how to overcome this issue?

currently as a workaround i'm skipping layers with groups > 1 but would really like to find a better solution,

thanks!

Hi

I have encountered this error while running the code. After getting the information about the filters which are to be pruned, while pruning the filters this issue occured.

/usr/local/lib/python2.7/dist-packages/torchvision/transforms/transforms.py:156: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
  "please use transforms.Resize instead.")
/usr/local/lib/python2.7/dist-packages/torchvision/transforms/transforms.py:397: UserWarning: The use of the transforms.RandomSizedCrop transform is deprecated, please use transforms.RandomResizedCrop instead.
  "please use transforms.RandomResizedCrop instead.")
Accuracy : 0.9248
Number of prunning iterations to reduce 67% filters 5
Ranking filters.. 
Layers that will be prunned {0: 5, 2: 4, 5: 7, 7: 6, 10: 25, 12: 24, 14: 19, 17: 42, 19: 50, 21: 66, 24: 72, 26: 74, 28: 118}
Prunning filters.. 
Traceback (most recent call last):
  File "finetune.py", line 270, in <module>
    fine_tuner.prune()
  File "finetune.py", line 228, in prune
    model = prune_vgg16_conv_layer(model, layer_index, filter_index)
  File "/disk2/jagadeesh/pytorch-pruning/prune.py", line 33, in prune_vgg16_conv_layer
    bias = conv.bias)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/conv.py", line 278, in __init__
    False, _pair(0), groups, bias)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/conv.py", line 34, in __init__
    if bias:
  File "/usr/local/lib/python2.7/dist-packages/torch/autograd/variable.py", line 125, in __bool__
    torch.typename(self.data) + " is ambiguous")
RuntimeError: bool value of Variable objects containing non-empty torch.FloatTensor is ambiguous```

Thanks

Hi,
I'm getting the following error while running pruning. Can anyone help me regarding this issue? It ran for one iteration out of 5 perfectly. It is giving an error in 2 iterations.

os: CentOS 7
torch (0.4.0)
torchvision (0.2.1)
python: 2.7

I downloaded the dog-cat dataset from kaggle, and run the
python finetune.py --train --train_path=. --test_path=.
Then I get the following Error:

$ python finetune.py --train --train_path=. --test_path=.
/home/web_server/dlpy72/dlpy/lib/python2.7/site-packages/torchvision/transforms/transforms.py:188: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
  "please use transforms.Resize instead.")
/home/web_server/dlpy72/dlpy/lib/python2.7/site-packages/torchvision/transforms/transforms.py:563: UserWarning: The use of the transforms.RandomSizedCrop transform is deprecated, please use transforms.RandomResizedCrop instead.
  "please use transforms.RandomResizedCrop instead.")
train data loading finished
Epoch:  0
THCudaCheck FAIL file=/pytorch/aten/src/THCUNN/generic/Threshold.cu line=67 error=59 : device-side assert triggered
Traceback (most recent call last):
  File "finetune.py", line 267, in <module>
    fine_tuner.train(epoches = 20)
  File "finetune.py", line 162, in train
    self.train_epoch(optimizer)
  File "finetune.py", line 180, in train_epoch
    self.train_batch(optimizer, batch.cuda(), label.cuda(), rank_filters)
  File "finetune.py", line 175, in train_batch
    self.criterion(self.model(input), Variable(label)).backward()
  File "/home/web_server/dlpy72/dlpy/lib/python2.7/site-packages/torch/tensor.py", line 93, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/web_server/dlpy72/dlpy/lib/python2.7/site-packages/torch/autograd/__init__.py", line 89, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: cuda runtime error (59) : device-side assert triggered at /pytorch/aten/src/THCUNN/generic/Threshold.cu:67
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [0,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [2,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [4,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [5,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [6,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [8,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [9,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [10,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [11,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [12,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [13,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [15,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [16,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [17,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [20,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [23,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [24,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [25,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [26,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [27,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [28,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [30,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [31,0,0] Assertion `t >= 0 && t < n_classes` failed.

What could be the cause and fix of this CUDNN_STATUS_BAD_PARAM error? I cannot get this to extract the features. Pytorch otherwise runs fine on my Ubuntu + Anaconda machine.

my@my:~/Dropbox/x/CV/pytorch-pruning$ CUDA_VISIBLE_DEVICES=0 python finetune_original.py --train
Downloading: "https://download.pytorch.org/models/vgg16-397923af.pth" to /home/my/.torch/models/vgg16-397923af.pth
100%|553433881/553433881 [08:01<00:00, 1148236.75it/s]
Epoch: 0
Traceback (most recent call last):
File "finetune.py", line 268, in
if args.train:
File "finetune.py", line 163, in train
print "Epoch: ", i
File "finetune.py", line 181, in train_epoch
for batch, label in self.train_data_loader:
File "finetune.py", line 176, in train_batch
else:
File "/home/my/anaconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 206, in call
result = self.forward(*input, **kwargs)
File "finetune.py", line 40, in forward
def forward(self, x):
File "/home/my/anaconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 206, in call
result = self.forward(*input, **kwargs)
File "/home/my/anaconda2/lib/python2.7/site-packages/torch/nn/modules/container.py", line 64, in forward
input = module(input)
File "/home/my/anaconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 206, in call
result = self.forward(*input, **kwargs)
File "/home/my/anaconda2/lib/python2.7/site-packages/torch/nn/modules/conv.py", line 237, in forward
self.padding, self.dilation, self.groups)
File "/home/my/anaconda2/lib/python2.7/site-packages/torch/nn/functional.py", line 40, in conv2d
return f(input, weight, bias)
RuntimeError: CUDNN_STATUS_BAD_PARAM

hello,i met a problem.what should train_path write? thank you

Has anyone tried pruning the SqueezeNet using this method and the program?
I have been trying to prune squeezenet but the test accuracy, during the finetuning after pruning first set of filter, is always 0.5, any idea what might be wrong?

I am confused about which filter to remove after getting the 'filter_index' from the 'compute_rank()' method.

Thank you!!!

I use Tesla k80 -12G *4,When I prunning the training test everything was normal, but after the pruned test memory overflow.
THCudaCheck FAIL file=/pytorch/aten/src/THC/generic/THCStorage.cu line=58 error=2 : out of memory
Traceback (most recent call last):
File "/home/b418-xiwei/.pycharm_helpers/pydev/pydevd.py", line 1664, in
main()
File "/home/b418-xiwei/.pycharm_helpers/pydev/pydevd.py", line 1658, in main
globals = debugger.run(setup['file'], None, None, is_module)
File "/home/b418-xiwei/.pycharm_helpers/pydev/pydevd.py", line 1068, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "/home/b418-xiwei/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/home/b418-xiwei/hgh/prune/finetune.py", line 343, in
fine_tuner.prune()
File "/home/b418-xiwei/hgh/prune/finetune.py", line 267, in prune
self.test()
File "/home/b418-xiwei/hgh/prune/finetune.py", line 187, in test
output = model(Variable(batch))
File "/home/b418-xiwei/anaconda3/envs/distiller/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/home/b418-xiwei/hgh/prune/finetune.py", line 68, in forward
x = self.features(x)
File "/home/b418-xiwei/anaconda3/envs/distiller/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/home/b418-xiwei/anaconda3/envs/distiller/lib/python3.6/site-packages/torch/nn/modules/container.py", line 91, in forward
input = module(input)
File "/home/b418-xiwei/anaconda3/envs/distiller/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/home/b418-xiwei/anaconda3/envs/distiller/lib/python3.6/site-packages/torch/nn/modules/pooling.py", line 142, in forward
self.return_indices)
File "/home/b418-xiwei/anaconda3/envs/distiller/lib/python3.6/site-packages/torch/nn/functional.py", line 360, in max_pool2d
ret = torch._C._nn.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
RuntimeError: cuda runtime error (2) : out of memory at /pytorch/aten/src/THC/generic/THCStorage.cu:58

I use batch_size=16,so it is

error dimension out of range

values =
torch.sum((activation * grad), dim = 0).
sum(dim=2).sum(dim=3)[0, :, 0, 0].data

My running environment is:

• Ubuntu 16.04LTS

• pytorch 0.4.1

• python 3.6

I got a problem is:
`/home/wangshuo/anaconda3/envs/pytorch/lib/python3.6/site-packages/torchvision-0.2.1-py3.6.egg/torchvision/transforms/transforms.py:188: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
/home/wangshuo/anaconda3/envs/pytorch/lib/python3.6/site-packages/torchvision-0.2.1-py3.6.egg/torchvision/transforms/transforms.py:563: UserWarning: The use of the transforms.RandomSizedCrop transform is deprecated, please use transforms.RandomResizedCrop instead.
Accuracy : 1.0
Number of prunning iterations to reduce 67% filters 5
Ranking filters..
Traceback (most recent call last):
File "finetune.py", line 270, in
fine_tuner.prune()
File "finetune.py", line 217, in prune
prune_targets = self.get_candidates_to_prune(num_filters_to_prune_per_iteration)
File "finetune.py", line 184, in get_candidates_to_prune
self.train_epoch(rank_filters = True)
File "finetune.py", line 179, in train_epoch
self.train_batch(optimizer, batch.cuda(), label.cuda(), rank_filters)
File "finetune.py", line 172, in train_batch
self.criterion(output, Variable(label)).backward()
File "/home/wangshuo/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/tensor.py", line 93, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/wangshuo/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/autograd/init.py", line 90, in backward
allow_unreachable=True) # allow_unreachable flag
File "finetune.py", line 77, in compute_rank
sum(dim=2).sum(dim=3)[0, :, 0, 0].data

RuntimeError: Dimension out of range (expected to be in range of [-2, 1], but got 3)
`
I have tried the solution given in https://github.com/jacobgil/pytorch-pruning/issues/6#issuecomment-332757962. This problem still exists.

Can anyone help me solve it?

Thanks!

Excuse me, about gradient I have some not understand, why the gradient shapes same as activation output shape. And the gradient is not weight gradient? it shape is [i , o , 3, 3]?

修建YOLOv2的话，剪掉一个卷积核都会使得准确率减为0是为什么呢？

Hello, I am new to github and pytorch. I do not know if my question is appropriate.

I have cloned the project to google colab and run the command line as instruction (python finetune.py --train) to verify the result. But it get an error that it cannot fine the directory to train function. If anyone had the same problem, could you held me out?

Thank you

In finetune.py, why do you have two different classes : FilterPrunner and PrunningFineTuner_VGG16 ? Isn't it possible to merge those two classes to have only one class ?

Thank you

Hi,

How long should I expect one pruning iteration to take? I'm talking about one iteration in this loop:
for layer_index, filter_index in prune_targets:
model = prune_vgg16_conv_layer(model, layer_index, filter_index)

It takes just over a second for me per iteration, is this normal? Can I move this process to the GPU since it seems idle at this stage?

I wonder if this pruning approach can be adapted to net like resnet

The given example of VGG16 works well for me. I want to try out this method on a MobiletNetV2 model. VGG16 has a pretty simple structure: a sequence of convs and other layers. However, MobiletNetV2 has a peculiar structure contains InvertedResidualBlocks, which is making the extension hard for me.

Any suggestions?

I don't see it stated anywhere
it is in prune.py

I'm looking to reduce the memory requirements for this model by 80%, if possible, to perform neural-style transfer on a mobile device. Will altering this script allow for such things?

when i use torch.load() to load pruned model , error happened: AttributeError: 'module' object has no attribute 'ModifiedVGG16Model', anyone meet this problem?

Hi, Jacob,

Seems the python script's indentation mixed tabs and spaces.

Running on the cats and dogs dataset, your repo does not work given the following error

my@my:~/Dropbox/x/CV/pytorch-pruning$ CUDA_VISIBLE_DEVICES=0 python finetune.py --train
PrunningFineTuner
('Train folder size', 25000)
('Test folder size', 25000)
fine_tuner.train()
Epoch: 0
Traceback (most recent call last):
File "finetune.py", line 273, in
fine_tuner.train(epoches = 3)
File "finetune.py", line 164, in train
self.train_epoch(optimizer)
File "finetune.py", line 181, in train_epoch
for batch, label in self.train_data_loader:
File "/home/my/anaconda2/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 212, in next
return self._process_next_batch(batch)
File "/home/my/anaconda2/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 239, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
RuntimeError: Traceback (most recent call last):
File "/home/my/anaconda2/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 41, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "/home/my/anaconda2/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 110, in default_collate
return [default_collate(samples) for samples in transposed]
File "/home/my/anaconda2/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 92, in default_collate
return torch.stack(batch, 0, out=out)
File "/home/my/anaconda2/lib/python2.7/site-packages/torch/functional.py", line 60, in stack
return torch.cat(inputs, dim, out=out)
RuntimeError: inconsistent tensor sizes at /py/conda-bld/pytorch_1493676237139/work/torch/lib/TH/generic/THTensorMath.c:2559

I trained the vgg and saved the model as pth file. then I load it for pruning some filters of it.
the last conv after pruning is not 512 anymore, some filters are gone.
how Pruning the last conv layer affects the first linear layer of the classifier which is (512 7 7, 4096).
how can I prune the input weights of classifier according to the last conv layer.