Comments (12)
You should be able. I mean you can send any binary blob (via the addAttachement method) to JS which can then be mapped to a proper typed array on the JS side.
In fact, you just need to use a memoryview
to wrap your np array.
from wslink.
Something like that should do the trick
return {
'data': self.addAttachement(memoryview(np_array),
'type': np_array.dtype,
}
And on the client:
const { data, type } = await rpcCall();
const array = new TYPE_MAPPING[type](data);
from wslink.
Hi, thanks for the example! Does this require a server on the client side (maybe that's a silly question , but just making sure)?
Also, I've been having issues installing wslink via npm for some reason, so is there just a download of the wslink.js compiled?
from wslink.
Also, I've been having issues installing wslink via npm for some reason, so is there just a download of the wslink.js compiled?
Sorry, this was because my internet connection was not good.
from wslink.
Thank you, I was able to get this working.
from wslink.
Hi again @jourdain, :)
On the server-side, in the wslink example there is this:
@exportRPC("myprotocol.image")
def image(self, alt = False):
filename = "kitware.jpg" if not alt else "kitware2.png"
filename = os.path.join(os.path.dirname(__file__), filename)
with open(filename, mode='rb') as file:
contents = file.read()
return { "blob": self.addAttachment(contents) }
to send the image as bytes.
On the client-side (js), it seems that you display the bytes like this:
function handleMessage(data) {
var blob = Array.isArray(data) ? data[0].blob : data.blob;
if (blob instanceof Blob) {
var canvas = document.getElementById('canvas_image');
var ctx = canvas.getContext('2d');
var img = new Image();
var reader = new FileReader();
reader.onload = function (e) {
img.onload = function () {
return ctx.drawImage(img, 0, 0);
};
img.src = e.target.result;
};
reader.readAsDataURL(blob);
} else {
log('result ' + blob);
}
}
I've been trying to figure out the fastest way to send a numpy array (image) and display it on a HTML canvas.
Would you be able to tell me what you use to send the numpy array to the js in wslink or at least point me in the right direction (whether it be a resource of portion of the wslink source)?
I've tried many different methods of doing this (base64 encoding, etc), but your wslink library does it the fastest from my tests. The issue is that wslink is too big of dependency for my needs and I probably won't need all of the lib's methods anyway. I've looked at the source, but haven't been able to figure out what addAttachment
does special to make JSON accept the bytes. (I must be missing something)
Also, you had mentioned wrapping the numpy array like memoryview(np_array)
And recieving it on the client side like:
const { data, type } = await rpcCall();
const array = new TYPE_MAPPING[type](data);
However, I haven't been able to get the memoryview or the bytes to send or receive correctly. (I guess the main thing that I wonder is what the addAttachment
does...or maybe it's elsewhere?)
Could you please help as it seems you've already implemented what I am looking for? Help is greatly appreciated. Thank you.
from wslink.
Do you want to send raw data or compressed data (png/jpg)? The example you provide rely on decompressing the image using the browser to decode the image hence all the reader/blob part.
If it is raw 4 bytes per pixel, you can just instantiate a Uint8ClampArray and use it to putImage directly to your canvas.
wslink is sending attachement as binary message over websocket (as binary message) but the receiving library is understanding that and unpack it properly inside the JS object that you get from the server.
I'm not sure what you are asking?
from wslink.
Thanks for you quick reply.
Do you want to send raw data or compressed data (png/jpg)? The example you provide rely on decompressing the image using the browser to decode the image hence all the reader/blob part.
My use-case is just displaying the processed numpy array image (preferably without artifacts) in the browser in an HTML canvas. That is what I am trying to achieve, with as little overhead as possible.
wslink is sending attachement as binary message over websocket (as binary message) but the receiving library is understanding that and unpack it properly inside the JS object that you get from the server.
I guess I was wondering how wslink did the binary sending/recieving, but if you have an idea of how numpy arrays can be done likewise that would be great.
My own solution (which wasn't fast enough for my liking) was to send the numpy array converted to btyes (np_array.tobytes()
) and loop through the array on the client side:
var recv_image_data = new Uint8Array(recv_data.data);
var canvas_image = document.getElementById('canvas_image');
var ctx = canvas_image.getContext('2d');
var imageData = ctx.createImageData(3648, 2736);
for (var i=0; i < imageData.data.length; i++){
imageData.data[i] = recv_image_data[i];
}
ctx.putImageData(imageData, 0, 0);
but that seems quite un-efficient and didn't seem to work for all array datatypes.
And it seems so, since my testing resulted in your method being much faster (though, that was with reading the bytes directly from the file like the wslink example).
Thank you for your time and patience. :)
from wslink.
What do you want to do next? Use wslink or not?
If wslink is a valid solution for you, did you managed to instantiate the proper typed array on the client side?
Then once you have the data on the client side, it might be easier to use vtk.js to deal with the rendering / field to color conversion for you on the GPU.
I'm not sure to understand where you are stuck with wslink?
from wslink.
What do you want to do next? Use wslink or not?
When I came across wslink, my thought was to use it. However, the dependency on the twisted package and others, etc when all I really need right now is a numpy array image displayed in the browser seems like too much. So, I am thinking that I will probably not use wslink for this case.
If wslink is a valid solution for you, did you managed to instantiate the proper typed array on the client side?
It seemed that the syntax of the wslink example and the code example you had provided were different so I wasn't 100% sure where to put that in the js, to be completely honest.
Then once you have the data on the client side, it might be easier to use vtk.js to deal with the rendering / field to color conversion for you on the GPU.
That could probably work (and it looks very promising), however I am thinking I should keep dependencies low and use only the websockets
library for this as I have already been trying to do. It seems that I am close to a solution, as I already have the code working, it's just that the method I am using is not as fast as I need it to be (which is why I was wondering what the method used in wslink is).
I'm not sure to understand where you are stuck with wslink?
I guess at this point, it would be very helpful if you could at least point me to the relevant portions in the wslink source for "communicating the binary attachments as a binary websocket message, avoiding the overhead of encoding and decoding", as it says in the wslink README.
I am not sure what it means for numpy arrays, but your help would invaluable for figuring this out, as I have spent a lot of hours looking and working on a solution already. It seems I'm close, but I haven't been able to deduce from the wslink source why the method is (way) faster than the method I am currently using.
I hope this clarifies a bit. I appreciate the hints you've already given for this. :)
from wslink.
Based on what you are saying it seems that you don't even need websocket and you can simply send your array (numpy) via an http endpoint in a binary format assuming the type you should transform that arraybuffer into is known.
But if you want the code snippet in wslink, here it is (but it is highly dependent of the dependency that you don't want).
from wslink.
BTW the code I gave you was for a numpy array. The example inside wslink get the raw bytes of a jpeg/png image and then use the browser reader to decode the jpg/png format. Since you don't have jpg/png encoding, you can convert the raw bytes into a single line in JS via array = new Float32Array(bytes)
assuming the bytes represent floats values (Hence the TYPE_MAPPING part)...
from wslink.
Related Issues (20)
- Handling ssl connexion HOT 5
- Server keeps running forever and trying to send messages to improperly disconnected clients HOT 4
- conda-forge package HOT 1
- Add Jupyter Comms as a supported backend HOT 6
- ImJoy-RPC backend? HOT 2
- RPC call application.exit.later unsuccessful: connection not open HOT 2
- Inheritance class WebAppServer from web.Application is discouraged HOT 2
- design.md Update HOT 1
- Invalid broadcast when servers are running as tasks HOT 1
- NameError: name 'pub' is not defined HOT 5
- Python 3.12 - dependence on aiohttp fails HOT 2
- Is it possible to support modifying the default mouse behavior? HOT 1
- Invalid f-string HOT 3
- aiohttp backend needs its own logger HOT 4
- WSLink examples are broken. HOT 3
- Increase max message size restriction HOT 7
- v2.1.0 introduce type hinting error for python < 3.10 HOT 1
- GET /paraview/292c5766-fa32-11eb-b91b-eb02855d4603 HOT 3
- Allow to launch wslink server without a shutdown timer HOT 3
- Remove secret from logs HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from wslink.