Sigmoid,tanh: TFL int16 reference code compared to float implementation is off by 3-4 bits about tflite-micro HOT 9 CLOSED

Hi. Can I just ask which code this issue refers to as the TFL int16 reference code.
If it is reference_ops::Tanh() that calls gemlowp::tanh() then this has a difference of 6 for sigmoid and 12 for tanh for int16 Q12 input.
However, I think TFLite no longer uses this for reference for int16 tanh and now uses reference_integer_ops::Tanh() that is more accurate, within I think +/-1 for int16 Q12
(for example https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/kernels/activations.cc#L945)
So, I do not think that TFLite int16 has an issue with Tanh/sigmoid precision.

from tflite-micro.

Assigning to myself, though I haven't had a chance to investigate this yet.

from tflite-micro.

This is a micro related issue,Please post this in micro repository and move this issue to closed status .Thanks!

from tflite-micro.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

from tflite-micro.

@petewarden

Can you please move this issue to tflite-micro github?

from tflite-micro.

@petewarden :A Gentle reminder.!

from tflite-micro.

@petewarden :A Gentle reminder.!

done

from tflite-micro.

Hi,
We tested reference_integer_ops::Logistic() & reference_integer_ops::Tanh() for all possible Q12 inputs.
reference_integer_ops::Logistic() is just 1 bit off for Q12 inputs.
But reference_integer_ops::Tanh() is 2 bit off for Q12 inputs and this difference increases from 2 bit to 4 bit when the input changes from Q12 to Q15.

from tflite-micro.

I dug a little deeper and found this:

https://github.com/tensorflow/tensorflow/blob/ee62dbd6d71243c0d7c79123f6b34d26cc048599/tensorflow/lite/kernels/activations_test.cc#L198-L215

So for Int16, 4 bits of error (2^-12) is actually expected for the TFLite implementation based on trade-off of accuracy and time/memory cost.

I will try to port the TFLite tests for int16 tanh and sigmoid (logistic) over to TFLM.

from tflite-micro.