zoujiu1 / lsqplus Goto Github PK

I'm wondering if there are some mistakes in this code. This function is to get the inital value about Scalebeta of activation_quantizer. Why need to get the gradiant about weight, and use it to update activation_quantizer's parameters. Is it more appropriate to change to weight = child.input ?
Looking forward to your reply.

https://github.com/ZouJiu1/LSQplus/blob/b8c6ea041d1678097c97cc77027e06c26d898d06/quantization/lsqplus_quantize_V2.py#L140C38-L140C38

朋友您好，这行代码应该是：

self.beta.data = self.beta.data*0.9 + 0.1* (mina - self.s.data * self.Qn)

evaluate.py中没有lsqprepare函数，如果把其改为lsqprepareV1然后加载训练好的模型进行评估，发现精度会非常低。
请问，在仅进行推断时，加载的模型和量化操作是不是和训练时有什么不同？

就是保存模型的时候为什么模型大小和没做量化时，大小一样

您好，您有试过LSQ+weight的per_channel模式吗，这个量化出来的效果很差，不如per_tensor的，发现出来的weight的scale比per_tensor的大一个数量级，您有思路解决这一问题吗

Hi author,
thank you for your great work!
I meet a problem when using Lsq+_V2:
Traceback (most recent call last): File "train.py", line 273, in <module> train(args) File "train.py", line 201, in train scaler.scale(loss).backward() File "/home/work/ssd1/anaconda3/envs/py38/lib/python3.8/site-packages/torch/_tensor.py", line 396, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs) File "/home/work/ssd1/anaconda3/envs/py38/lib/python3.8/site-packages/torch/autograd/__init__.py", line 173, in backward Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass RuntimeError: Function ALSQPlusBackward returned an invalid gradient at index 2 - got [1] but expected shape compatible with [0]
It seems that the backward func of ALSQPlus has some errors, any advice of how to solve this problem?
Thanks in advance!

I've tried to convert LSQ, LSQ+ model to ONNX model but I got a runtime error

This is the code what I tried to run

import torch
from brevitas.export import export_onnx_qcdq

export_onnx_qcdq(quantized_model, torch.randn(2048,52).cuda(), export_path='lsq+_.onnx')

here is the error message that I got from converting LSQ+ model to onnx
First error line
"name": "RuntimeError",
"message": "ONNX export failed: Couldn't export Python operator ALSQPlus\n\nDefined at:\ne:\LSQplus-master\LSQplus-master\quantization\lsqplus_quantize_V2.py(154):

Last error lines
e:\LSQplus-master\LSQplus-master\quantization\lsqplus_quantize_V2.py:154:0\n %onnx::Gemm_67 : Float(*, , strides=[16, 1], requires_grad=0, device=cuda:0) = ^WLSQPlus(0.007394637578467616, -128, 127, False)(%model_fp32.output.0.weight, %model_fp32.output.0.weight_quantizer.s) # e:\LSQplus-master\LSQplus-master\quantization\lsqplus_quantize_V2.py:221:0\n %68 : Float(, *, strides=[9, 1], requires_grad=0, device=cuda:0) = onnx::Gemm[alpha=1., beta=1., transB=1](%onnx::Gemm_66, %onnx::Gemm_67, %model_fp32.output.0.bias) # e:\LSQplus-master\LSQplus-master\quantization\lsqplus_quantize_V2.py:319:0\n return (%68)\n"
}

Have you ever tried this before? or It isn't supported from onnx itself? I wonder about it

你好，感谢你的开源工作；
我直接用了一下你的代码运行了一下，选择的是lsq v1版本的，可以看到你的scale参数都是在量化器forward的时候用nn.parameter处理的，但是你的trains.py中给optimizer添加参数组时，你是用的所有需要梯度的参数，这个时候并没有将量化器的scale参数囊括进去，这也就直接导致后面训练的时候scale参数将一直是常量，并没有通过训练进行学习；
尽管在forward中将scale参数给nn.parameter处理了，但是这些参数正如前面所说没有添加到optimizer中，所以相当于这部分参数没有添加到计算图中；

如果我的理解有误的话，请及时告知，谢谢！

Hi，thanks for your excellent work.
after train model with LSQplus, how to export S/Z files? like aimet export .encondings file? then be used in snpe

“model = DDP(model, device_ids=[LOCAL_RANK], output_device=LOCAL_RANK,find_unused_parameters= False)” get stuck and can not run next code.

def update_LSQplus_activation_Scalebeta(model):
    for name, child in model.named_children():
        if isinstance(child, (QuantConv2d, QuantConvTranspose2d, QuantLinear)):
            weight = child.weight.data
            s = child.activation_quantizer.s.data
            beta = child.activation_quantizer.beta.data
            Qn = child.activation_quantizer.Qn
            Qp = child.activation_quantizer.Qp
            g = child.activation_quantizer.g
            # print('before: ', name, child.activation_quantizer.s.grad.data, child.activation_quantizer.beta.grad.data, s, beta)
            q_w = (weight - beta) / s
            # print(q_w)
            smaller = (q_w < Qn).float() #bool值转浮点值，1.0或者0.0
            bigger = (q_w > Qp).float() #bool值转浮点值，1.0或者0.0
            between = 1.0 - smaller -bigger #得到位于量化区间的index
            grad_alpha = ((smaller * Qn + bigger * Qp + 
                           between * Round.apply(q_w) - between * q_w) * g).sum().unsqueeze(dim=0)
            grad_beta = ((smaller + bigger) * g).sum().unsqueeze(dim=0)
            # print('grad_beta: ',grad_beta,g, smaller.sum(), bigger.sum(), between.sum(),Qn, Qp)
            child.activation_quantizer.s.grad.data.add_(g*(2*(child.quant_input-child.input)*grad_alpha).sum().unsqueeze(dim=0))
            child.activation_quantizer.beta.grad.data.add_(g*(2*(child.quant_input-child.input)*grad_beta).sum().unsqueeze(dim=0))

• lsqplus_quantize_V1.py#L213
• 为什么 grad_alpha 求的是 conv weight 关于 s 的导数，而不是 quant_input 关于 s 的导数？是我哪里理解错了，还是这里有个 bug ?

在使用lsq+在进行detection的时候，遇到一个问题

请问您有遇到过这种情况吗？看起来是backward的时候有个变量没有梯度？

Thank you so much for your open source work!! This project really help me out a lot. I just have one question regarding LSQ and LSQ+. I hope I am not misunderstanding anything, but as far as I concern, the learned step size should be a scale value. However, during training, I noticed that there were times where the step size will become negative and the quantized performance of the model will become really bad. Is this some kinds of bug from the original paper or is it me who is making a huge mistake. Any help from you is appreciated, thank you!

我在训练前后使用torch.nn.module.extra_repr打印了尺度和偏移的相关信息，但是发现与训练前无异
但是，我尝试只训练权重，而不改变偏移尺度等信息，得到的精度确不如通过lsq+训练得到的结果
说明训练起到了效果，但是打印的信息确没有变化
我不清楚为什么 “打印了尺度和偏移的相关信息，但是发现与训练前无异”
希望得到您的回复

Thanks for your open source work. There is an initalization variable self.g in lsqqlus_quantiza_V1.py:L138, I'm wondering the usage of it.
Looking forward to your reply.