Comments (2)
There is a couple of reasons. The main one is that NB2 actually contains some architectures that do not make sense - in their codebase if a cell structure is, for example, a simple conv followed by none then: 1) conv is still executed, 2) none takes as input the output of conv, and simply ignores it by producing torch.zeros_like(x)
, 3) the rest of the network is executed normally and the network is trained even though it is effectively learning nonsense (as input does not flow to the output). For measuring latency we optimise models by removing unnecessary operations, which also means that we simply do not execute networks like that at all (as they are constant functions anyway, after training). From the top of my head, I would say that's the reason behind all the "15284" pickles. For the remaining ones, I would suspect the extra missing results to be models that failed to run on-device, despite our best effort - unfortunately I don't have much time now to take a close look at those. If you want to help me out a bit, I would appreciate if you could identify what archs are missing. Otherwise I will put it in my backlog and will take a look at them when the amount of other work decreases a bit. iirc, some models failed to run/convert with SNPE, although SNPE has been updated significantly since then so it might not be reproducible now. also edge-tpu had problems with at some some models - if these are in fact the files that are missing some models, that's most likely the reason.
from eagle.
Thanks a lot for your response! That makes sense.
I will try to identify which ones are missing soon, and get back to you. Hope thats okay! I already have the script ready so it shouldn't be too much work.
from eagle.
Related Issues (15)
- Bug Report HOT 1
- Accuracy Results Meaning? HOT 3
- Latency Dataset
- Get rid of `pickle` HOT 5
- Is the dataset in google drive still available? HOT 4
- Is accuracy dataset uploaded? HOT 3
- Can you share the pre-trained desktop GPU latency predictor and FLOPS predictor? HOT 2
- Number of devices HOT 2
- Request regarding the DARTS and NB101 spaces HOT 4
- Use with --load (instead of --transfer) to eval pretrained model. HOT 1
- "Not enough points" error HOT 3
- View models? HOT 8
- pip install eagle: python3 or python2? HOT 3
- ModuleNotFoundError: No module named 'eagle' HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from eagle.