Comments (11)
Thanks for offering help! Honestly, I'm not sure we are ready to accept contributions, but let me try suggesting some good first tasks.
I think the current ONNX does not have F.shift
mentioned in #67 and the process will be somewhat involved. As the first op, it'd be probably better to start with an ONNX standard op which has the standard test. Looking at the ONNX's operators (https://github.com/onnx/onnx/blob/master/docs/Operators.md), I guess followings could be a good candiates:
- HardSigmoid
- IsNaN: (note perhaps it's better to support new attributes proposed in: onnx/onnx#1884)
- Tile
As for HardSigmoid, I think this op can be implemented by other ONNX ops. For such cases, I'm trying to use compiler/simplifier.cc to keep the backend implementation simple. 7505e17 could be a good example.
As for the latter two ops, I think you need to define new XCVM ops and call ChainerX ops in the backend implementation. See 2cc5b43 for an example.
As for CH2O/elichika: since we're thinking of switching to elichika from CH2O, we'd like more ops support in elichika. The process would be something like
- Copy a file from ch2o/tests/node to elichika/tests/node (e.g., Sigmoid.py)
- Modify the test and scripts/elichika_tests.py
- Modify elichika/elichika/chainer2onnx.py and elichika/elichika/parser/functions_builtin.py until the new test passes.
However, currently, adding a new op is as simple as CH2O. I'm guessing we need to refactor how operator emitters are implemented in future, so I'm not very sure if adding new ops to elichika with the current interface is a great idea. @durswd, please correct me if I'm wrong.
from chainer-compiler.
Thanks for the detailed information! I will start with the ops you mentioned.
Also, I was just wondering, is onnx-chainer not involved in any part of the process of supporting new operations?
from chainer-compiler.
Sorry I forgot mentioning onnx-chainer. C++ chainer-compiler is an ONNX importer while onnx-chainer, CH2O, and elichika are ONNX exporters. For end-to-end user experience (export chainer model as an ONNX file and run it by chainer-compiler), adding new ops to onnx-chainer is important. So contributions are welcome. As you mentioned, chainer/onnx-chainer#44 will be a good example, though API was changed a bit since then. However, I'm not sure if there is a good easy op to be implemented. ONNX-chainer has a longer history than this project and it's coverage is fairly good. FYI, Where
op, which was recently added to ONNX, is missing (chainer/onnx-chainer#24) but I think @disktnk is working on it now.
From perspective of C++ chainer-compiler, when you add support of an op, we need a test ONNX model which contains the op. The easiest way is to use ONNX's standard tests. For three ops I mentioned, ONNX has test cases (https://github.com/onnx/onnx/tree/master/onnx/backend/test/data/node/test_hardsigmoid). You can find it in third_party/onnx/onnx/backend/test/data/node/test_hardsigmoid in your cloned repository. When there is no ONNX's standard testcase (e.g., non-standard ops) or it's coverage is not 100% (e.g., backprop), we need to generate tests by ourselves. For such case, we use scripts/gen_backprop_tests_oc.py (onnx-chainer based test generator), scripts/gen_backprop_tests_pc.py (ch2o based test generator), or scripts/gen_extra_tests.py.
from chainer-compiler.
Hi @shinh
This makes things much more clear! I still have one more question though.
Since ch2o is an ONNX exporter, from what I have understood, I think that the func.py file contains a mapping from Chainer functions to ONNX ops. While it seems straightforward to implement those functions which have a similar counterpart in ONNX ops (such as F.expand_dims -> unsqueeze), I couldn't comprehend ONNX op names such as 'ChainerSequenceStack' or 'ChainerResizeImages' and how they are themselves defined.
from chainer-compiler.
We added some custom ops to ONNX. I think they are either
- sequence related (to handle Python's list)
- gradient related (ONNX does not have MaxPoolGrad for example)
- ops missing in ONNX (e.g., ROIAverageAlign2D)
I think we should/want to propose some ops in the category 3, but unfortunately, we haven't done yet :(
Though there's no documentation about these custom ops, here is the list: https://github.com/pfnet-research/chainer-compiler/blob/master/compiler/gen_node.py#L212
Also, note other vendors have some custom ops (e.g., https://github.com/Microsoft/onnxruntime/blob/master/docs/ContribOperators.md).
from chainer-compiler.
@vermashresth Please let me hijack this issue to make sure information is not scattered.
Questions from Rishav:
- I need some starting pointers for understanding the experimental chainer compiler, like where to start from and what major libraries I need to be familiar with.
I think the former question is mostly answered in #101 (comment)
As for major libraries, I think you first need to understand ONNX. I think these two documents are very helpful to familialize yourself to ONNX.
https://github.com/onnx/onnx/blob/master/docs/IR.md
https://github.com/onnx/onnx/blob/master/docs/Operators.md
- An example of a recent expansion added to the compiler so that I can follow what is the expectation for the project.
may be hints to help you to understand how operators are created and executed.
from chainer-compiler.
@shinh Thank you for addressing my questions. I find myself interested in extending the processing of python jump statements in both elichika and ch2o. I am aware of how dreadful using tf.while_loop with break condition is. Extending onnx graph generation to support loops with jumps would be an awesome feature I think.
I have some more queries relating to elichika and ch2o.
- From overview docs, I understand that currently elichika is meant to eventually replace ch2o. If that's the case, what is the current gap between them in terms of
- ONNX operator coverage--what subset of operators can the onnx output contain?
- supported Chainer functions and nodes---is the list of tests(node, syntax and model) an accurate representation of supported operations in each module? If not, how can I identify disparity.
- Elichika seems to be missing the ability to form backward computation computation graph. I don't understand how the backward computation graph is being generated in ch2o from the chainer model. Wouldn't it require processing the source of backward call too?
- In elichika parser core#L21-L23, search is not performed in child modules which poses a limitation on how the modules need to be imported. For eg.
import chainer.functions as F
andfrom chainer import functions
make the module functions available in search, but if it's indirectly imported via chainer likeimport chainer
orimport chainer.functions
then the search fails. Implementation can be improved to do a recursive search on the parent modules instead. - If say my chainer model uses F.softmax_cross_entropy() in forward function, does the translation to onnx via either elichila and ch2o involve just creating a node with name '@f.<lineno.>.softmax_cross_entropy', type 'Softmax' and input-output names? Am I missing something?
from chainer-compiler.
tf.while_loop
Yeah, my experience with it was exactly the motivation I wanted Python loop support in the compiler stack.
ONNX operator coverage
Yes, you are right. (ch2o|elichika)/tests/node should be almost accurate. You would see elichika really needs more ops :)
backward in elichika
Both CH2O and elichika do not emit graphs with backprop. The compiler middle end generates op nodes for backward computation. (https://github.com/pfnet-research/chainer-compiler/blob/master/compiler/gradient.cc and https://github.com/pfnet-research/chainer-compiler/blob/master/compiler/gradient_ops.cc)
The commented out code was meant to generate expected gradient values by running the test model with Chainer. It was just a TODO, and it was fairly easy to fix: #124
module lookup
I'm not 100% sure, but I think you are right (@durswd would correct me if I'm wrong). In general, I think there are a bunch of rooms for improvement around name lookup in elichika. Patches are welcome :)
functions to ONNX translation
Yes, you are right. It should be very easy to add ops once we implemented basic framework to handle variety of ops.
As for the specific case of F.softmax_cross_entropy
, ONNX's Softmax
is F.softmax
, not F.softmax_cross_entropy
. ONNX does not have it so we need either
- implement it by some consecutive ops (https://github.com/chainer/onnx-chainer/blob/master/onnx_chainer/functions/loss.py#L35) or
- use custom ChainerSoftmaxCrossEntropy op (https://github.com/pfnet-research/chainer-compiler/blob/master/ch2o/ch2o/funcs.py#L215)
Also, in addition to inputs/outputs, you also need to set attributes properly. For example, ONNX's Softmax
has axis
attribute so this value should be copied from the parameter passed to F.softmax
. https://github.com/pfnet-research/chainer-compiler/blob/master/ch2o/ch2o/funcs.py#L533
from chainer-compiler.
@shinh I have the a doubt regarding ForAndIf.py syntax test in ch2o. It seems from the netron visualization of the generated onnx file that the if condition computation graph is not being represented.
I even tried the official net drawer tool, and it seems it also doesn't completely represent the graph (though does give more details).
I want to see the details of loop subgraph. Is there a way to do so?
Also, what does _find_in_out() function do exactly? I understand that it is keeping track of values will change inside the for loop body. But there are very specific condition based statements inside that I an unable to comprehend.
from chainer-compiler.
I think these visualizers do not support subgraphs, unfortunately. Usually, I just read ASCII proto dump from build/tools/dump
or build/tools/run_onnx --dump_onnx
, but extracting subgraphs by this script may also help: https://github.com/shinh/test/blob/master/extract_onnx_subgraph.py
from chainer-compiler.
Yeah, _find_in_out
is very complex and would still have some bugs. I think a lot of complexity were due to CH2O is a one pass translator. It does not know which variables are used/modified/created inside a for-loop when it is parsing Python for
. _find_in_out
tries to extract these information using information in subgraphs and Env.(read|wrote)_attrs
.
from chainer-compiler.
Related Issues (20)
- Reconsider the API to access Graph
- Dead code removal in canonicalizer.
- Unify how flags/configs are organized
- GemmOp which directly uses clblas
- Resize needs some fixes
- Compile MaskRCNNFPNResNet50 with elichika
- RoiAlign batch_indices type handled incorrectly HOT 1
- Extend FLOPS estimator for ConvTranspose and ConvGradWeight HOT 1
- Deprecate CH2O HOT 1
- Run Mask R-CNN R-50-FPN in model zoo HOT 7
- Expose chainer-compiler flags feature to menoh API
- Missing menoh_make_model_data_from_onnx_data_on_memory HOT 3
- Update gast to 0.3.x HOT 2
- dtype and passed std::vector<T> data type may not match HOT 2
- Deprecation of TensorRT output formula
- Honor opset_import in ModelProto HOT 2
- Make menoh version resnet50.onnx run
- Mac bot flaky?
- Generate JUnit XML in run_onnx for CI test failure tracking
- Update ChainerX when fast `IndexIterator` is merged
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from chainer-compiler.