furiosa-ai / furiosa-sdk Goto Github PK

View Code? Open in Web Editor NEW

29.0 29.0 8.0 130.19 MB

FuriosaAI SDK

License: Apache License 2.0

Python 91.55% Makefile 1.43% Batchfile 0.31% CMake 0.66% C++ 0.01% C 5.70% Shell 0.07% Dockerfile 0.28%

furiosa-sdk's People

Stargazers

Watchers

Forkers

hyunsik justinhmkim kstreee-furiosa ileixe hyeokjunejeon seoyoung-furiosa aurotripathy jjsair0412

furiosa-sdk's Issues

Session::run, AsyncSession::submit should check input shapes

see the title. Currently, the inference API relies on the check of native library's tensor size. For better error handling, Python runtime should check input shapes.

Use 'pip install' with single quote in documents

In zsh, bracket means globing. So, pip install furiosa-sdk[full] will not work in zsh. So, we have to recommend users to use pip install 'furiosa-sdk[full]' in our documents.

session should log the npu device name when initializing the device.

As the title, session should print out the log message about what NPU device is opened.

Introduce int64 type to python sdk

The native library recently introduced int64 type (https://github.com/furiosa-ai/npu-tools/pull/2433). Python SDK also needs to introduce int64 type.

runtime should expose more understandable errors

In many cases, Session returns internal errors when it faces compilation error or errors which cannot be handled. We need to improve this behavior of furiosa-sdk-runtime.

furiosa-registry / model error 처리 추가

Error 처리를 exception throw로 처리하지 않고 검증해서 처리 (https://github.com/rustedpy/result 참고)

Validate 실행 결과

성공
Passed!

실패

(furiosa) root@901b1c76ca8c:/toolchains# furiosa validate mobilenetv2-7.onnx 
Failed. The result has been written to validation.txt
(furiosa) root@901b1c76ca8c:/toolchains# cat validation.txt 
Stdout:
[Step 1] Checking if the model can be transformed into a quantized model ...

Stderr:
INFO:furiosa_sdk_runtime._api.v1:successfully loaded dynamic library libnux.so.1.0.0
[Step 1] Failed

Traceback (most recent call last):
  File "/root/miniconda3/envs/furiosa/bin/furiosa-validate", line 12, in <module>
    sys.exit(main())
  File "/root/miniconda3/envs/furiosa/lib/python3.8/site-packages/furiosa_sdk_model_validator/__init__.py", line 69, in main
    validate(Path(args.model_path))
  File "/root/miniconda3/envs/furiosa/lib/python3.8/site-packages/furiosa_sdk_model_validator/__init__.py", line 44, in validate
    raise e
  File "/root/miniconda3/envs/furiosa/lib/python3.8/site-packages/furiosa_sdk_model_validator/__init__.py", line 37, in validate
    quantized_model = post_training_quantization_with_random_calibration(model=onnx.load_model(model_path),
  File "/root/miniconda3/envs/furiosa/lib/python3.8/site-packages/onnx/__init__.py", line 119, in load_model
    model = load_model_from_string(s, format=format)
  File "/root/miniconda3/envs/furiosa/lib/python3.8/site-packages/onnx/__init__.py", line 156, in load_model_from_string
    return _deserialize(s, ModelProto())
  File "/root/miniconda3/envs/furiosa/lib/python3.8/site-packages/onnx/__init__.py", line 97, in _deserialize
    decoded = cast(Optional[int], proto.ParseFromString(s))
google.protobuf.message.DecodeError: Error parsing message

unit tests occassionally fails because floating point comparison fails

======================================================================
FAIL: test_async_session (test_mnist_mobinenet_v2.MnistMobilenetV2)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jenkins/agent/workspace/furiosa-sdk-tests/furiosa-sdk/python/furiosa-sdk-runtime/tests/test_mnist_mobinenet_v2.py", line 28, in test_async_session
    tester.assert_equals(np.array(self.x_test[idx:idx + 1], dtype=np.uint8))
  File "/home/jenkins/agent/workspace/furiosa-sdk-tests/furiosa-sdk/python/furiosa-sdk-runtime/tests/test_base.py", line 60, in assert_equals
    assert_tensors_equal(tf_results, nux_results)
  File "/home/jenkins/agent/workspace/furiosa-sdk-tests/furiosa-sdk/python/furiosa-sdk-runtime/tests/test_base.py", line 20, in assert_tensors_equal
    assert np.array_equal(expected, result), "{} was expected, but the result was {}".format(expected, result)
AssertionError: [[[[211 119 118  53  74  96 123 102  91 111]]]] was expected, but the result was [[[[211 119 118  53  74  96 123 102  90 111]]]]

Investigate the best way to generate Sphinx document from C header

Background

Currently, we are using Doxygen (https://www.doxygen.nl/index.html) to generate C reference document. Please visit the generated C reference at https://furiosa-ai.github.io/renegade-manual/references/nux/. Like pdoc3 issue, the page also looks different from our official documentation at https://furiosa-ai.github.io/renegade-manual/sdk/latest/ko/ generated by Sphinx (https://www.sphinx-doc.org).

What we should do

The ultimate goal is to generate Sphinx documents for C reference. Please investigate if there is a way to generate Sphinx documents for C reference. If it's not possible, please find alternative way like bridging between Doxygen and Sphinx. It would be great if someone does some experiments with our C header and show how it works.

The following articles are what I've found so far:

Unhashable type when CompletionQueue::recv fails

See the error message. We need to handle the error case.

Traceback (most recent call last):
  File "/home/jenkins/agent/workspace/furiosa-sdk-tests/furiosa-sdk/python/furiosa-sdk-runtime/tests/test_async_session.py", line 33, in test_run_async
    for key, _ in self.tester.queue:
  File "/home/jenkins/agent/workspace/furiosa-sdk-tests/furiosa-sdk/python/furiosa-sdk-runtime/furiosa_sdk_runtime/session.py", line 160, in __next__
    return self.recv()
  File "/home/jenkins/agent/workspace/furiosa-sdk-tests/furiosa-sdk/python/furiosa-sdk-runtime/furiosa_sdk_runtime/session.py", line 135, in recv
    raise into_exception(err)
  File "/home/jenkins/agent/workspace/furiosa-sdk-tests/furiosa-sdk/python/furiosa-sdk-runtime/furiosa_sdk_runtime/errors.py", line 159, in into_exception
    if err in _errors_to_exceptions.keys():
TypeError: unhashable type

ctrl + c cannot interrupt the compilation

When users try to 'ctrl + c', session.create is not interrupted if it is compiling a model. This is because the interruption is not passed to C API. We need to fix it.

(venv) root@jason-fpga:~/workspace/furiosa-sdk/examples/furiosa-sdk-runtime# python image_segmentation.py ../assets/images/segmentation_4.jpg 
Inference using the existing quantized model
libnpu.so --- v2.0, built @ 32f26b7
INFO:furiosa_sdk_runtime._api.v1:successfully loaded dynamic library libnux.so.1.0.0
num_slices: 64
[1/6]  Compiling from onnx to dfg
Done in 0.28828996s                                  
[2/6]  Compiling from dfg to ldfg                 <- even if a user try to 'ctrl + c' here
▪▪▪▪▪ [1/3] Splitting graph...Done in 75.11472s
▪▪▪▪▪ [2/3] Lowering...Done in 23.57374s
▪▪▪▪▪ [3/3] Precalculating operators...Done in 13.817107s
Done in 112.52346s
[3/6]  Compiling from ldfg to cdfg
Done in 0.001491236s
[4/6]  Compiling from cdfg to gir
Done in 1.2123256s
[5/6]  Compiling from gir to lir
▪▪▪▪▪ [1/2] Operator scheduling...Done in 0.15889277s
▪▪▪▪▪ [2/2] Memory allocating...Done in 0.65708476s
Done in 0.82257146s
[6/6]  Compiling from lir to enf
Traceback (most recent call last):
  File "image_segmentation.py", line 121, in <module>
    run_segmentation(str(image_path.resolve()), str(model_path.resolve()), is_fp32)
  File "image_segmentation.py", line 74, in run_segmentation
    with session.create(str(model_path)) as sess:
  File "/root/workspace/furiosa-sdk/examples/furiosa-sdk-runtime/venv/lib/python3.8/site-packages/furiosa_sdk_runtime/session.py", line 248, in create
    return Session(model)
  File "/root/workspace/furiosa-sdk/examples/furiosa-sdk-runtime/venv/lib/python3.8/site-packages/furiosa_sdk_runtime/session.py", line 46, in __init__
    err = LIBNUX.nux_session_create(model_image, len(model_image), options, byref(sess))
KeyboardInterrupt       <- It can be interrupted after C API is done.

UnboundLocalError: local variable 'libnux_path' referenced before assignment

While running test program, the following exception is occured.

Traceback (most recent call last):
  File "random_input_inference.py", line 9, in <module>
    from furiosa_sdk_runtime.tensor import DataType
  File "/home/furiosa-demo/furiosa-sdk/venv/lib/python3.8/site-packages/furiosa_sdk_runtime/__init__.py", line 4, in <module>
    from furiosa_sdk_runtime import session, model, tensor, errors
  File "/home/furiosa-demo/furiosa-sdk/venv/lib/python3.8/site-packages/furiosa_sdk_runtime/session.py", line 9, in <module>
    from ._api import LIBNUX
  File "/home/furiosa-demo/furiosa-sdk/venv/lib/python3.8/site-packages/furiosa_sdk_runtime/_api/__init__.py", line 5, in <module>
    from .v1 import LIBNUX
  File "/home/furiosa-demo/furiosa-sdk/venv/lib/python3.8/site-packages/furiosa_sdk_runtime/_api/v1.py", line 77, in <module>
    LIBNUX = _find_native_libs()
  File "/home/furiosa-demo/furiosa-sdk/venv/lib/python3.8/site-packages/furiosa_sdk_runtime/_api/v1.py", line 57, in _find_native_libs
    libnux_path = _find_user_libs()
  File "/home/furiosa-demo/furiosa-sdk/venv/lib/python3.8/site-packages/furiosa_sdk_runtime/_api/v1.py", line 38, in _find_user_libs
    return libnux_path
UnboundLocalError: local variable 'libnux_path' referenced before assignment

Implement scripts to bump up the versions

We need to manually update the versions in setup.py and setup.cfg in each module everytime we release. Let's make a script to automate it.

Change the python reference generator to Sphinx

Background

Currently, we are using pdoc3 (https://pdoc3.github.io/pdoc/) to generate our python reference. Please see the generated reference at https://furiosa-ai.github.io/renegade-manual/references/python/. The page looks different from our official documentation at https://furiosa-ai.github.io/renegade-manual/sdk/latest/ko/ generated by Sphinx (https://www.sphinx-doc.org).

What we should do

It would be great if we migrate pdoc3 to Sphinx in order to have more consistent document look and use only one document generation system. You can refer to the following documents about how to generate python docstring using Sphinx:

fp32 onnx model을 session에 넣었을 때, error message

Session에 fp32 onnx model을 넣을 경우, 현재 sdk에서는 에러를 출력합니다.
다만 에러 메시지가 다소 비직관적이어서 개선하면 좋을 것 같습니다.
현재 에러메시지는 다음과 같습니다.

root@hanjoon:~/furiosa-sdk/examples/furiosa-sdk-runtime# ./random_input_inference.py ../assets/fp32_models/MobileNetV2_10c_10d.onnx
INFO:furiosa_sdk_runtime._api.v1:successfully loaded dynamic library libnux.so.1.0.0
Loading and compiling the model ../assets/fp32_models/MobileNetV2_10c_10d.onnx
num_slices: 64
num_slices: 64
[1/6]  Compiling from onnx to dfg
thread '<unnamed>' panicked at 'no entry found for key', crates/npu-ir/src/furiosa_ir/furiosa_ir_onnx.rs:2647:55
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: Any', crates/nux/src/session/mod.rs:108:10
Traceback (most recent call last):
  File "./random_input_inference.py", line 53, in <module>
    random_input_inference(model_path, num_inf=num_inf)
  File "./random_input_inference.py", line 16, in random_input_inference
    with session.create(str(model_path)) as sess:
  File "/usr/local/lib/python3.8/dist-packages/furiosa_sdk_runtime/session.py", line 248, in create
    return Session(model)
  File "/usr/local/lib/python3.8/dist-packages/furiosa_sdk_runtime/session.py", line 48, in __init__
    raise into_exception(err)
furiosa_sdk_runtime.errors.InternalError: unknown (native error code: 15)

furiosa-ai / furiosa-sdk Goto Github PK

furiosa-sdk's People

Stargazers

Watchers

Forkers

furiosa-sdk's Issues

Background

What we should do

Background

What we should do

Recommend Projects

Recommend Topics

Recommend Org