Comments (9)
Thanks for your interest in CMSIS-NN. We shall update the header description for that as part of #108.
Are you suggesting a similar check as here https://github.com/tensorflow/tensorflow/blob/451a94246d83d171b446141ec40569f6a571f7df/tensorflow/lite/kernels/internal/common.cc#L25 ?
from cmsis-nn.
The use case I was thinking of is when I know the multiplier is 24 bits (e.g. comes from a binary32 floating point).
Then the product fits on 32 bits and I don't want rounding at all.
Conceptually it could be done using a shift value of 31: product would be "left shifted by one" before adding the rounding bit and then right shifted by one.
I can also "pre shift" my multiplier to obtain the same result with a shift value of 30, so no problem of enforcing shift values lower than 30, but then it is not consistent with the #else
branch which computes a sensible result for shift = 31.
from cmsis-nn.
Actually I think the shift
parameter does not have the same signification depending on whether or not the function is compiled with CMSIS_NN_USE_SINGLE_ROUNDING
.
In the "single rounding" case, the biased shift applies to the result which is computed exactly.
In the other case, the input is shifted before performing the product.
As a result it is, if I am not mistaken, actually impossible to access the product low bits in the second case.
I don't know if this is intended, but that is rather counterintuitive and not well documented.
from cmsis-nn.
You mentioned in the other case (double rounding case) that the input is shifted before performing the product.
That is only done if shift is positive. Also for the same case the divide by (2 ^ abs(shift)) is only done if shift is negative. Perhaps documentation could be made more clear about that.
Also to be clear the output may be slightly different between the two branches and that is intentional.
Furthermore the intention is that both branches have the same numerical behavior as for tensorflow as stated here.
from cmsis-nn.
I expected the result to vary slightly, but not that much.
Let's imagine my multiplier value is 2.
Product bits of interest are on bits [1-32].
To "cancel" the implicit 31 bit right shift I need to set shift to 30.
But because the val * (1 << LEFT_SHIFT(shift))
results in an int32_t
, in the double rounding case I will lose 30 bits of value before computing the product, while in the single rounding case it will be OK.
Of course I can change my multiplier to 1 << 30
and have a shift of only 1 to get the same result, and it will work in the two cases.
If this is expected it should probably be documented (I don't see it on the tensorflow int8 quantization specification either).
The USE_SINGLE_ROUNDING
macro name is in this case a bit misleading.
Something like EXACT_PROD_BEFORE_REQUANTIZE
or EXTENDED_PROD_BEFORE_REQUANTIZE
would make the difference more clear in my opinion.
from cmsis-nn.
So now you want to change name of the macro CMSIS_NN_USE_SINGLE_ROUNDING to something like EXACT_PROD_BEFORE_REQUANTIZE or EXTENDED_PROD_BEFORE_REQUANTIZE ?
Would it not be better to keep a similar name as in tensorflow ?
from cmsis-nn.
I think it depends on what is the goal of the library: if the end purpose is to be an implementation backend for tensorflow, then sticking to their naming even if they are misleading makes sense.
If the goal is to provide library that can be used directly outside from tensorflow, then I think it makes sense to detach from it.
from cmsis-nn.
Everyone is free to use CMSIS-NN in their project. However it is currently only bit-exact to TensorFlow reference kernels. If defining CMSIS_NN_USE_SINGLE_ROUNDING it is no longer bit-exact unless also specifying TFLITE_SINGLE_ROUNDING in tensorflow.
The name of the flag does not prevent CMSIS-NN to be used by a project..
from cmsis-nn.
I am not saying that the name of the flag is preventing anyone from using the library.
Just that it is rather misleading, as in practice (for the requantize case, I haven't checked other usages) the value difference can be way bigger than just double rounding issues.
As documentation is not very clear on it, user needs to know that actually this name does not refers to what it does, but is a reference to some Tensorflow internal define.
However it is currently only bit-exact to TensorFlow reference kernels.
Based on that it seems that it is really tied to tensorflow.
A less tied library might rather aim at numerical accuracy, not specifically bitwise equality.
Thank you for the clarifications!
from cmsis-nn.
Related Issues (20)
- missing arm_s8_to_s16_unordered_with_offset.c when I included the pack HOT 3
- Output diff for new lstm-operator
- Confusing doc on `arm_nn_requantize`
- MVE Conv 1 x N does not handle all configurations HOT 1
- Unit tests fails to build with -O0
- Inconsistency of CMSIS-NN Quantization Method(Q-format) with ARM Documentation HOT 4
- Inconsistency of CMSIS-NN Quantization Method(Q-format) with ARM Documentation HOT 2
- No MVE support Conv2D int16
- Move test platform to other FVPs HOT 6
- IAR compiler issues HOT 1
- Is there a roadmap? HOT 3
- Does the MVEI version of CMSIS NN have implementations of q15 and q7? HOT 1
- Vector of variable length used in arm_convolve_1x1_s8_fast HOT 1
- Documentation warning HOT 1
- What does CMSIS exactly refer to? HOT 1
- Occur an ASM error when compiling arm_nn_mat_mul_core_4x_s8 HOT 11
- reserved identifier violation HOT 6
- Is there some easy to get started guide? HOT 1
- Include/Internal/arm_nn_compiler.h: undefined __ARM_FEATURE_MVE HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cmsis-nn.