Git Product home page Git Product logo

torch-js's Introduction

TorchJS

npm version CI publish-prebuild

TorchJS is a JS binding for PyTorch. Its primary objective is to allow running TorchScript inside Node.js program. Complete binding of libtorch is possible but is out-of-scope at the moment.

This fork is maintained by Techainer

Changes after fork

  • Add support for List (Javascript Array), Dict (Javascript Object), String, float (Javascript number) as inputs and outputs.

  • Add CUDA support.

  • Add ops from torchvision.

  • Add async support for forward function.

  • Add async support for call_scripted_function(function_name, **inputs) to call any method of the scripted module.

  • Provide prebuild binary for cross NVIDIA GPU from Pascal to Ampere

  • Fixed some weird conversion arround List[Tensor] input.

  • Updated libtorch to 1.8.1 and torchvision to 0.9.1

Install

To install the forked version, you can install it from npm:

yarn add torch-js@npm:@techainer1t/torch-js

Example

In tests/resources/torch_module.py, you will find the defination of our test module and the code to generate the trace file.

class TestModule(torch.nn.Module):
    def __init__(self):
        super(TestModule, self).__init__()

    def forward(self, input1, input2):
        return input1 + input2

Once you have the trace file, it may be loaded into NodeJS like this

const torch = require("torch-js");
const modelPath = `test_model.pt`;
const model = new torch.ScriptModule(testModelPath);
const inputA = torch.rand([1, 5]);
const inputB = torch.rand([1, 5]);
const res = await model.forward(inputA, inputB);

More examples regarding tensor creation, ScriptModule operations, and loading models can be found in our examples folder.

torch-js's People

Contributors

arition avatar b21quocbao avatar kittipatv avatar lamhoangtung avatar raghavmecheri avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

b21quocbao

torch-js's Issues

Missing some binary in distributed prebuilt

Error message indicate this bug:

Error: libgomp-7c85b1e2.so.1: cannot open shared object file: No such file or directory

libgomp-7c85b1e2.so.1 was available in /torch-js/build/libtorch/lib/libgomp-a34b3233.so.1 when build from source and when unzip libtorch downloaded from pytorch.org. But some how cmake-js and prebuilt does not include any .so.1 file with the distributed prebuilt

A temporary workarround on Linux is to create some symlink:

/usr/lib/x86_64-linux-gnu/libgomp-7c85b1e2.so.1 -> /usr/lib/x86_64-linux-gnu/libgomp.so.1
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudart-6d56b25a.so.11.0 -> /usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudart.so.11.0
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libnvToolsExt-24de1d56.so.1 -> /usr/local/cuda-11.1/targets/x86_64-linux/lib/libnvToolsExt.so.1

Also 1080Ti also having this issue:

libcudart.so.10.2: cannot open shared object file: No such file or directory

Will investigate later cc @b21quocbao

Empty Tensor with Simple Add Operation

Why am I getting an empty tensor after this operation ?

const torch = require("torch-js");

async function test() {
    const modelPath = "model.pt";
    const model = new torch.ScriptModule(modelPath);
    const inputA = torch.rand([1, 5]);
    const inputB = torch.rand([1, 5]);
    const res = await model.forward(inputA, inputB);

    return res
}

test().then(result => console.log(result))

and here is my torch module:

import torch

class TestModule(torch.nn.Module):
    def __init__(self):
        super(TestModule, self).__init__()

    def forward(self, input1, input2):
        return input1 + input2


def main():
    model = TestModule()

    sample_input1 = torch.rand(1,5)
    sample_input2 = torch.rand(1,5)

    module = torch.jit.trace(model, (sample_input1, sample_input2))

    torch.jit.save(module, "model.pt")

if __name__ == "__main__":
    main()

Missing builtin op torchvision::nms in Windows

Here is the full log

const scriptModule = new torch.ScriptModule(`models/retinaface-torch/scripted_model_${haveGpu ? 'gpu' : 'cpu'}_19042021.pt`);
                     ^

Error:
Unknown builtin op: torchvision::nms.
Could not find any similar ops to torchvision::nms. This op may not exist or may not be currently supported in TorchScript.
:
  File "/home/geneous/anaconda3/envs/main/lib/python3.7/site-packages/torchvision/ops/boxes.py", line 36
    """
    _assert_has_ops()
    return torch.ops.torchvision.nms(boxes, scores, iou_threshold)
           ~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
Serialized   File "code/__torch__/torchvision/ops/boxes.py", line 26
  _8 = __torch__.torchvision.extension._assert_has_ops
  _9 = _8()
  _10 = ops.torchvision.nms(boxes, scores, iou_threshold)
        ~~~~~~~~~~~~~~~~~~~ <--- HERE
  return _10
'nms' is being compiled since it was called from 'batched_nms'
  File "/home/geneous/anaconda3/envs/main/lib/python3.7/site-packages/torchvision/ops/boxes.py", line 75
        offsets = idxs.to(boxes) * (max_coordinate + torch.tensor(1).to(boxes))
        boxes_for_nms = boxes + offsets[:, None]
        keep = nms(boxes_for_nms, scores, iou_threshold)
               ~~~ <--- HERE
        return keep
Serialized   File "code/__torch__/torchvision/ops/boxes.py", line 18
    _7 = torch.slice(offsets, 0, 0, 9223372036854775807, 1)
    boxes_for_nms = torch.add(boxes, torch.unsqueeze(_7, 1), alpha=1)
    keep = __torch__.torchvision.ops.boxes.nms(boxes_for_nms, scores, iou_threshold, )
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
    _0 = keep
  return _0
'batched_nms' is being compiled since it was called from 'RetinaNetDetector.postprocess_batch'
Serialized   File "code/__torch__/model.py", line 139
    landms: Tensor) -> List[Tuple[List[Tensor], List[Tensor]]]:
    _41 = __torch__.net.apply_indices_2d_to_3d
    _42 = __torch__.torchvision.ops.boxes.batched_nms
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
    batch_size, channels, im_height, im_width, = torch.size(batched_tensor)
    image_size_str = torch.format("{}x{}", im_height, im_width)
'RetinaNetDetector.postprocess_batch' is being compiled since it was called from 'RetinaNetDetector.forward_batch'
  File "/media/geneous/e3359753-18af-45fb-b3d4-c5a0ced00e0c/Techainer/face/retinaface_torchscript/model.py", line 461
        # Postprocessing
        # st = time.time()
        output = self.postprocess_batch(batched_tensor, all_scale, loc, conf, landms)
                 ~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
        # torch.cuda.synchronize()
        # print("Postprocess tooks:", (time.time()-st)*1000, "ms")
Serialized   File "code/__torch__/model.py", line 43
    _2 = (self.model).forward(batched_tensor, )
    loc, conf, landms, = _2
    output = (self).postprocess_batch(batched_tensor, all_scale, loc, conf, landms, )
                                                                            ~~~~~~ <--- HERE
    return output
  def postprocess(self: __torch__.model.RetinaNetDetector,

    at Object.<anonymous> (C:\Users\ngoba\work\backend-monolith\test.js:3:22)
    at Module._compile (internal/modules/cjs/loader.js:999:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
    at Module.load (internal/modules/cjs/loader.js:863:32)
    at Function.Module._load (internal/modules/cjs/loader.js:708:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12)
    at internal/main/run_main_module.js:17:47

Expected a value of type 'List[List[Tensor]]' for argument 'list_bboxes' but instead found type 'List[List[Tensor]]'

I had fixed a similar issue for List[Tensor], but haven't find a way to fix this yet. Created this issue mainly as a reminder.

node:44) UnhandledPromiseRejectionWarning: Error: predict_batch() Expected a value of type 'List[List[Tensor]]' for argument 'list_bboxes' but instead found type 'List[List[Tensor]]'.
Position: 2
Declaration: predict_batch(__torch__.main.Liveness_mbv3 self, Tensor[] images, Tensor[][] list_bboxes, float r_padding=0.25, int max_batchsize=1024) -> (Tensor[])
Exception raised from checkArg at /pytorch/aten/src/ATen/core/function_schema_inl.h:162 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x42 (0x7f9bc8cd62f2 in /horus/node_modules/@lamhoangtung/torch-js/build/libtorch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::string const&) + 0x5b (0x7f9bc8cd367b in /horus/node_modules/@lamhoangtung/torch-js/build/libtorch/lib/libc10.so)
frame #2: <unknown function> + 0xd9668f (0x7f9bb6fb568f in /horus/node_modules/@lamhoangtung/torch-js/build/libtorch/lib/libtorch_cpu.so)
frame #3: torch::jit::GraphFunction::operator()(std::vector<c10::IValue, std::allocator<c10::IValue> >, std::unordered_map<std::string, c10::IValue, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::pair<std::string const, c10::IValue> > > const&) + 0x2d (0x7f9bb945b88d in /horus/node_modules/@lamhoangtung/torch-js/build/libtorch/lib/libtorch_cpu.so)
frame #4: torch::jit::Method::operator()(std::vector<c10::IValue, std::allocator<c10::IValue> >, std::unordered_map<std::string, c10::IValue, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::pair<std::string const, c10::IValue> > > const&) + 0x138 (0x7f9bb94691e8 in /horus/node_modules/@lamhoangtung/torch-js/build/libtorch/lib/libtorch_cpu.so)
frame #5: <unknown function> + 0x115f3 (0x7f9bc8f2a5f3 in /torch-js/build/Release/torch-js.node)
frame #6: Napi::AsyncWorker::OnAsyncWorkExecute(napi_env__*, void*) + 0x5e (0x7f9bc8f2e35e in /torch-js/build/Release/torch-js.node)
frame #7: /usr/local/bin/node() [0x132e72e]
frame #8: <unknown function> + 0x76db (0x7f9c073b26db in /lib/x86_64-linux-gnu/libpthread.so.0)
frame #9: clone + 0x3f (0x7f9c070dba3f in /lib/x86_64-linux-gnu/libc.so.6)

 

(node:44) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:44) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.