Suitable for this use-case? about wslink HOT 12 CLOSED

You should be able. I mean you can send any binary blob (via the addAttachement method) to JS which can then be mapped to a proper typed array on the JS side.
In fact, you just need to use a memoryview to wrap your np array.

from wslink.

Something like that should do the trick

return {
   'data': self.addAttachement(memoryview(np_array),
   'type': np_array.dtype,
}

And on the client:

const { data, type } = await rpcCall();
const array = new TYPE_MAPPING[type](data);

from wslink.

Hi, thanks for the example! Does this require a server on the client side (maybe that's a silly question , but just making sure)?

Also, I've been having issues installing wslink via npm for some reason, so is there just a download of the wslink.js compiled?

from wslink.

Also, I've been having issues installing wslink via npm for some reason, so is there just a download of the wslink.js compiled?

Sorry, this was because my internet connection was not good.

from wslink.

Thank you, I was able to get this working.

from wslink.

Hi again @jourdain, :)

On the server-side, in the wslink example there is this:

    @exportRPC("myprotocol.image")
    def image(self, alt = False):
        filename = "kitware.jpg" if not alt else "kitware2.png"
        filename = os.path.join(os.path.dirname(__file__), filename)
        with open(filename, mode='rb') as file:
            contents = file.read()
            return { "blob": self.addAttachment(contents) }

to send the image as bytes.

On the client-side (js), it seems that you display the bytes like this:

function handleMessage(data) {
    var blob = Array.isArray(data) ? data[0].blob : data.blob;

    if (blob instanceof Blob) {
      var canvas = document.getElementById('canvas_image');
      var ctx = canvas.getContext('2d');
      var img = new Image();
      var reader = new FileReader();

      reader.onload = function (e) {
        img.onload = function () {
          return ctx.drawImage(img, 0, 0);
        };

        img.src = e.target.result;
      };

      reader.readAsDataURL(blob);
    } else {
      log('result ' + blob);
    }
  }

I've been trying to figure out the fastest way to send a numpy array (image) and display it on a HTML canvas.

Would you be able to tell me what you use to send the numpy array to the js in wslink or at least point me in the right direction (whether it be a resource of portion of the wslink source)?

I've tried many different methods of doing this (base64 encoding, etc), but your wslink library does it the fastest from my tests. The issue is that wslink is too big of dependency for my needs and I probably won't need all of the lib's methods anyway. I've looked at the source, but haven't been able to figure out what addAttachment does special to make JSON accept the bytes. (I must be missing something)

Also, you had mentioned wrapping the numpy array like memoryview(np_array)

And recieving it on the client side like:

const { data, type } = await rpcCall();
const array = new TYPE_MAPPING[type](data);

However, I haven't been able to get the memoryview or the bytes to send or receive correctly. (I guess the main thing that I wonder is what the addAttachment does...or maybe it's elsewhere?)

Could you please help as it seems you've already implemented what I am looking for? Help is greatly appreciated. Thank you.

from wslink.

Do you want to send raw data or compressed data (png/jpg)? The example you provide rely on decompressing the image using the browser to decode the image hence all the reader/blob part.

If it is raw 4 bytes per pixel, you can just instantiate a Uint8ClampArray and use it to putImage directly to your canvas.

wslink is sending attachement as binary message over websocket (as binary message) but the receiving library is understanding that and unpack it properly inside the JS object that you get from the server.

I'm not sure what you are asking?

from wslink.

Thanks for you quick reply.

Do you want to send raw data or compressed data (png/jpg)? The example you provide rely on decompressing the image using the browser to decode the image hence all the reader/blob part.

My use-case is just displaying the processed numpy array image (preferably without artifacts) in the browser in an HTML canvas. That is what I am trying to achieve, with as little overhead as possible.

wslink is sending attachement as binary message over websocket (as binary message) but the receiving library is understanding that and unpack it properly inside the JS object that you get from the server.

I guess I was wondering how wslink did the binary sending/recieving, but if you have an idea of how numpy arrays can be done likewise that would be great.

My own solution (which wasn't fast enough for my liking) was to send the numpy array converted to btyes (np_array.tobytes()) and loop through the array on the client side:

  var recv_image_data = new Uint8Array(recv_data.data);

  var canvas_image = document.getElementById('canvas_image');
  var ctx = canvas_image.getContext('2d');

    var imageData = ctx.createImageData(3648, 2736);
    for (var i=0; i < imageData.data.length; i++){
      imageData.data[i] = recv_image_data[i];
    }
    ctx.putImageData(imageData, 0, 0);

but that seems quite un-efficient and didn't seem to work for all array datatypes.

And it seems so, since my testing resulted in your method being much faster (though, that was with reading the bytes directly from the file like the wslink example).

Thank you for your time and patience. :)

from wslink.

What do you want to do next? Use wslink or not?

If wslink is a valid solution for you, did you managed to instantiate the proper typed array on the client side?

Then once you have the data on the client side, it might be easier to use vtk.js to deal with the rendering / field to color conversion for you on the GPU.

I'm not sure to understand where you are stuck with wslink?

from wslink.

What do you want to do next? Use wslink or not?

When I came across wslink, my thought was to use it. However, the dependency on the twisted package and others, etc when all I really need right now is a numpy array image displayed in the browser seems like too much. So, I am thinking that I will probably not use wslink for this case.

If wslink is a valid solution for you, did you managed to instantiate the proper typed array on the client side?

It seemed that the syntax of the wslink example and the code example you had provided were different so I wasn't 100% sure where to put that in the js, to be completely honest.

Then once you have the data on the client side, it might be easier to use vtk.js to deal with the rendering / field to color conversion for you on the GPU.

That could probably work (and it looks very promising), however I am thinking I should keep dependencies low and use only the websockets library for this as I have already been trying to do. It seems that I am close to a solution, as I already have the code working, it's just that the method I am using is not as fast as I need it to be (which is why I was wondering what the method used in wslink is).

I'm not sure to understand where you are stuck with wslink?

I guess at this point, it would be very helpful if you could at least point me to the relevant portions in the wslink source for "communicating the binary attachments as a binary websocket message, avoiding the overhead of encoding and decoding", as it says in the wslink README.

I am not sure what it means for numpy arrays, but your help would invaluable for figuring this out, as I have spent a lot of hours looking and working on a solution already. It seems I'm close, but I haven't been able to deduce from the wslink source why the method is (way) faster than the method I am currently using.

I hope this clarifies a bit. I appreciate the hints you've already given for this. :)

from wslink.

Based on what you are saying it seems that you don't even need websocket and you can simply send your array (numpy) via an http endpoint in a binary format assuming the type you should transform that arraybuffer into is known.

But if you want the code snippet in wslink, here it is (but it is highly dependent of the dependency that you don't want).

from wslink.

BTW the code I gave you was for a numpy array. The example inside wslink get the raw bytes of a jpeg/png image and then use the browser reader to decode the jpg/png format. Since you don't have jpg/png encoding, you can convert the raw bytes into a single line in JS via array = new Float32Array(bytes) assuming the bytes represent floats values (Hence the TYPE_MAPPING part)...

from wslink.