Bug when set ENABLE_BIAS_QUANT = True about brevitas HOT 5 CLOSED

Hello,
You need to pass a QuantTensor to the QuantConv2d layer, which happens when you enable return_quant_tensor=True in the activation before that QuantConv2d layer. For the very first conv layer, you should insert a quantized identity (a quantized hard tanh) right at the beginning of the network.

As a side note, I see you are working on a quantized MelGAN implementation. GANs can be quite tricky to quantize. We have a working 8 bit version internally that we are going to release at some point in the next couple of months.

Alessandro

from brevitas.

@volcacius, yeah, i see. I was successful to quantize melgan to float16 on tflite, it's run 2x faster than realtime. On 8 bit, the accuracy drop much :)), there are many white noise :v. i'm still investigate ur implementation and tflite implementation. Somehow the output of tflite and your's framework is different on 8bit :D. (32bit and 16bit is almost same). If u know the difference of ur quantize and tflite procedure, pls let me know :'(. I thought it was because of the bias when i don't use fake-quantize aware for it, but remove bias didn't solved the problem :D

from brevitas.

It really depends on how you are setting up the quantized layers.
In general TFLite is a great tool for production-oriented quantization, while Brevitas is oriented towards research, which is why it provides many more options.
For what it's worth, our internal results at 8 bit with MelGAN on LJSpeech are on par with floating point quality. I'll be happy to share the details once the model is release.

from brevitas.

@volcacius Looking forward to ur model :D . Noticed that my result on LJspeech using 8bit (base on this framework) are on par with float32 too (use pytorch) but have some difference when convert to tflite :D. BTW, thanks for ur great implementation again :D.

from brevitas.

I see now, glad to hear about your good results and thanks for the positive feedback! Please cite us if you plan to release/publish them somewhere, I would really appreciate.
On the export side, unfortunately there aren't any plans for a TFLite compatible flow at the moment. We are working on a custom ONNX based flow, but it's going to target deployment on our own FPGAs.

from brevitas.