Converting regular Keras weights to Qkeras about qkeras HOT 10 CLOSED

Hi @xhlulu

Thank you very much for the question. Sorry, we do not have the document (yet).

Besides the tips from @nunescoelho , for now, you may follow the example/example_keras_to_qkeras.py to convert model to its quantized counterpart. For example, the model_quantize usage

And use qmodel.load_weights("Your_weight_file") just like what people usually do.

If you want to see the weights, you might want to try
model_save_quantized_weights from here

for layer in qmodel.layers:
  try:
    if layer.get_quantizers():
      q_w_pairs = zip(layer.get_quantizers(), layer.get_weights())
      for _, (quantizer, weight) in enumerate(q_w_pairs):
        qweight = K.eval(quantizer(weight))
        print("quantized weight")
        print(qweight)
  except AttributeError:
    print("warning, the weight is not quantized in the layer %s", layer.name)

@nunescoelho it could be helpful that we can document the process.

from qkeras.

Remember that QKeras layers only change the behavior of the forward pass (that's the straight through estimator), so look at the functions:

• model_save_quantized_weights
• model_quantize

In particular, model_quantize at end transfers the weights from the original keras model to the quantized model (optionally).

And remember as a rule of thumb, every arithmetic operation needs to be followed by a quantizer if you want the QKeras model to mimic any implementation.

from qkeras.

I was also applying QKeras to a pretrained model, but unfortunately my validation accuracy was very low after using model_quantize.
Is Retraining necessary, so can QKeras only be used for quantization-aware training? Because it seems to me that loading a pretrained model and quantizing it (post-training quantization) does not work.
And is there any documentation on using QKeras for pretrained models so far?

Best regards and many thanks,
asti205

from qkeras.

@asti205 retraining is necessary or you can try to train from qkeras directly by modifying your keras model into qkeras version.

from qkeras.

from qkeras.

Hello @nunescoelho ,

thank you for the explanation, but that is clear to me :)

The reason why I was asking is, that I could get a validation accuracy that was magnitudes higher using the TFLite converter. So I was explicitly interested in post-training quantization. However, I could also get a much higher accuracy with an own quantizer that is approximating the TFLite-converter behaviour.

Also, I think it is not the best option to set the post-comma bitwidths fixed, it is better to quantize adaptively depending on the actual weights. At least that is what I found out from the analysis of the TFLite Converter.

Best regards,
asti205

from qkeras.

hey, i want to know. in our win10, i ues pip install qkeras error.i want to install qkeras package what should i do ?

from qkeras.

Hi @Sejudyblues thank you for the question!

I assume you could find the package via pip install qkeras, right?

Could you try to clone the repo and use python setup.py install to install this package? BTW, we have not tried in win10.

from qkeras.

Since it has been quite for a while, I closed this issue. Feel free to reopen it.

from qkeras.

Hi, I looking for a tool for post-training inference with custom option(like bit-width). I use tensorflow and keras for my project, I trained a model and pruned it with tfmot, then I want to quantize it to a full integer model. Since retraining will change the sparsity of the model, I don't want to use quantization-aware-training. The other way is tf-lite, but it doesn't support quantize with custom option. Finally, I look for this project. Can I use this project to solve my problem? And I learned that full integer quantization needs represent dataset to learn some thing for activation quantization, does any functions in this project can do that? Or can I change source code to eliminate back-propagation in training and just use training to get parameter for activation quantization.

Thanks!

from qkeras.