kig / datastream.js Goto Github PK
View Code? Open in Web Editor NEWDataStream.js is a library for reading data from ArrayBuffers
DataStream.js is a library for reading data from ArrayBuffers
for some strange reason functions incorrectly assume that arr.byteOffset
is 0.
When using Chrome, I encountered range limits when using readString on large binary objects.
In order to get around these errors I modified the readString method as follows. I haven't 100% tested this but I figured I would share in case anyone else encounters these issues:
DataStream.prototype.readString = function(length, encoding) {
if (encoding == null || encoding == "ASCII") {
try{
var str = "";
var arr = this.mapUint8Array(length == null ? this.byteLength-this.position : length);
var length = arr.length;
for (var i=0; i < length; i++){
str += String.fromCharCode(arr[i])
}
//return String.fromCharCode.apply(null,arr);
return str
}
catch(err){
console.error("ERROR " + err.message)
}
} else {
return (new TextDecoder(encoding)).decode(this.mapUint8Array(length));
}
};
This is a great utility! I'd like to parse a series of variable-size items into an array using the readStruct method. The number of elements isn't known; only the total size of the segment containing the items is known. It should read as many items as possible without exceeding the segment size. Is this possible with the current DataStream.js implementation? Also, can you add licensing information? Thanks!
Not sure how I would do this with the DataStream object.
In python / C I would do something like
class CCSDS_PriHdr_t_StreamId (my_endian):
"""Some comments"""
_pack_ = 1
_fields_ = [ ("packet_version_number" , ctypes.c_ushort ,3),
("packet_type" , ctypes.c_ushort ,1),
("secondary_header_present" , ctypes.c_ushort ,1),
("apid" , ctypes.c_ushort ,11)]
This would create a 16 bit ctype structure with 4 fields of bits 3,1,1,11.
Trying to stream data from an embedded device and then decode it in a browser.
Setup:
install node js
install requirejs module (sudo npm install requirejs)
Steps:
Create a file (example.js) with the following code in the same dir as your DataStream.js file:
(function(requirejs){
requirejs(['./DataStream'],function(){
var ds = new DataStream();
});
})(require('requirejs'));
run the file: node example.js
Result:
Crashes with the following error.
RangeError: byteOffset out of range.
at Object.defineProperty.set (... /DataStream.js:108:24)
at Object.DataStream (../DataStream.js:19:17)
at example.js:8:18
It seems that node cannot handle declaring a 0 byte ArrayBuffer
Workaround/Fix:
In line 19 of DataStream.js
set the minimum ArrayBuffer size to 1
Change
this.buffer = new ArrayBuffer(arrayBuffer || 0);
to
this.buffer = new ArrayBuffer(arrayBuffer || 1);
Currently, it's unclear what licence this repo is under.
Hi,
Opera 12 is little buggy with typed arrays, BYTES_PER_ELEMENT is not defined in prototypes, so some functions in ds.js are not working. Simplest workaround (possibly the only and most simple bugfix) is to define it in prototypes like this:
if (Uint8Array.prototype.BYTES_PER_ELEMENT == undefined) {
Uint8Array.prototype.BYTES_PER_ELEMENT = Uint8Array.BYTES_PER_ELEMENT;
Int8Array.prototype.BYTES_PER_ELEMENT = Int8Array.BYTES_PER_ELEMENT;
Uint8ClampedArray.prototype.BYTES_PER_ELEMENT = Uint8ClampedArray.BYTES_PER_ELEMENT;
Uint16Array.prototype.BYTES_PER_ELEMENT = Uint16Array.BYTES_PER_ELEMENT;
Int16Array.prototype.BYTES_PER_ELEMENT = Int16Array.BYTES_PER_ELEMENT;
Uint32Array.prototype.BYTES_PER_ELEMENT = Uint32Array.BYTES_PER_ELEMENT;
Int32Array.prototype.BYTES_PER_ELEMENT = Int32Array.BYTES_PER_ELEMENT;
Float64Array.prototype.BYTES_PER_ELEMENT = Float64Array.BYTES_PER_ELEMENT;
}
Hello kig,
First of all, I would like to say that DataStream is a fundamental piece of software, a must ! Many, many thanks to you and for all your works (HTML5 Rocks !). From our point of view, we should say
the approach you have choosen is the best one.
We plan to use DataStream for validating upload of files in a production environment, and, maybe, for creating efficient storage of datas. But, we nead a little help.
testTypeArray
routine : line 232 throws an error :var arr = new this[t+"Array"](ds.byteLength/elen);
var arr = new this[t+"Array"]( Math.floor(ds.byteLength/elen) );
if (ds.byteOffset & rarr.BYTES_PER_ELEMENT === 0) {...}
Furthermore, I can't understand why you perform a this._realloc(len * N);
at the beginning of all the map functions : in my eyes they are not really necessary.
Besides 'seek' and 'isEo' api's, you could add at no charge :
function (saut){return this.seek(this.position+saut);},
function (){return this.position;}
("where".length=5 chars ! - sorry)Best regards and thank you for the time that you will devote ourselves
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.