The pytorch-weight-prune from zepx

pytorch-weight-prune's People

Contributors

Stargazers

Watchers

pytorch-weight-prune's Issues

why run_example.sh cannot run??

Multiplication of mask 2 times in layers

Hi, I'm relatively new to deep learning area, and studying network pruning methods. This repository have helped me a lot to understand key ideas of pruning. Thank you :)

By the way, I have one question on the masked layer classes.

When setting the mask, the weight matrix of the layer is multiplicated with mask. However, it is multiplicated once more on forward. More specifically, these lines.

on set_mask: self.weight.data = self.weight.data*mask_var.data

on forward: weight = self.weight * mask_var

Is it on purpose?, if yes, could you explain why?

I'll wait for the reply. Have a joyful day!

class MaskedLinear(nn.Linear):
    def __init__(self, in_features, out_features, bias=True):
        super(MaskedLinear, self).__init__(in_features, out_features, bias)
        self.mask_flag = False
    
    def set_mask(self, mask):
        self.register_buffer('mask', mask)
        mask_var = self.get_mask()
        self.weight.data = self.weight.data*mask_var.data
        self.mask_flag = True
    
    def get_mask(self):
        # print(self.mask_flag)
        return to_var(self.mask, requires_grad=False)
    
    def forward(self, x):
        if self.mask_flag == True:
            mask_var = self.get_mask()
            weight = self.weight * mask_var
            return F.linear(x, weight, self.bias)
        else:
            return F.linear(x, self.weight, self.bias)

Recommend Projects