clayzermk1 / node-marshal Goto Github PK
View Code? Open in Web Editor NEWParse Ruby's Marshal strings into JavaScript objects/JSON
License: MIT License
Parse Ruby's Marshal strings into JavaScript objects/JSON
License: MIT License
Hi,
when I want to try deserialize a rails 2.3 session data with some customized key inside the data.
var sdata = new Buffer(session_data, 'base64');
var m = new Marshal(sdata);
I got
MarshalError: unsupported typecode 108 (index: 57, hex: 6c, utf8: l)
After further investigation in ruby 1.8 marshal.c, seems that l is for 'long' (?) which I think it's the equivalent of bignum in ruby. And I know the data inside is integer (Time.now.to_i), I don't why ruby/rails treat it as bignum when serializing it. If you can add support for bignum that would be great.
Hi there, thanks for this useful library!
It seems that there might be a problem with parsing Bignum
s, specifically in the context of a hash. For reasons I don't understand, I can't actually make a test where it fails - but I can make it fail just fine in the REPL. Consider the following:
2.2.4 :045 > Marshal.dump({1=>1499769350, 2=>2}).unpack('H*')
=> ["04087b0769066c2b0706aa645969076907"]
> var Marshal = require('marshal')
undefined
> var m = new Marshal
undefined
> m.load(new Buffer("04087b0769066c2b0706aa645969076907", 'hex')).parsed
MarshalError: unsupported typecode 43 (index: 7, hex: 2b, utf8: +)
at Marshal._parse (/home/dvangeest/projects/test/node_modules/marshal/lib/marshal.js:52:15)
at Marshal._parseHash (/home/dvangeest/projects/test/node_modules/marshal/lib/marshal.js:170:20)
at Marshal._parse (/home/dvangeest/projects/test/node_modules/marshal/lib/marshal.js:45:21)
at Marshal.load (/home/dvangeest/projects/test/node_modules/marshal/lib/marshal.js:264:26)
at repl:1:3
at sigintHandlersWrap (vm.js:22:35)
at sigintHandlersWrap (vm.js:96:12)
at ContextifyScript.Script.runInThisContext (vm.js:21:12)
at REPLServer.defaultEval (repl.js:346:29)
at bound (domain.js:280:14)
I tried unsuccessfully to add a unit test for this case... it passes, which makes me wonder if a) I'm just doing something wrong, or b) if there's a problem with the test framework.
Based on my limited understanding of Ruby's Marshal format, it looks like the index is not being advanced beyond the typecode for Bignum
. Therefore, the +
following the l
, denoting that it's a positive number, is read as the next type code. My JS is not good enough to spot the error, if indeed there is one.
Any ideas?
👋 this endpoint /api/v1/dependencies states:
Returns a marshalled array of hashes for all versions of given gems. Each hash contains a gem version with its dependencies making this useful for resolving dependencies.
I'm buffering the response into a Buffer and then I'm passing it to this module as new Marshal(buf)
However, it throws an error:
MarshalError: unsupported typecode 8 (index: 2, hex: 08, utf8:)
at Marshal._parse (/node_modules/marshal/lib/marshal.js:54:15)
at Marshal.load (/node_modules/marshal/lib/marshal.js:345:26)
at new Marshal (/node_modules/marshal/lib/marshal.js:13:33)
I can provide you with a simple snippet that reproduces the issue.
The following string throws an exception at parse time:
'\u0004\b{\nI"\u000fsession_id\u0006:\u0006ETI"%621c43c4db9732740e2fafd941635f0a\u0006;\u0000TI"\u0010_csrf_token\u0006;\u0000FI"1REC7hnjqnd0mNgEzS0+dYTEzUY7NIL1WbAjMiiHv5kc=\u0006;\u0000FI"\u0019warden.user.user.key\u0006;\u0000T[\bI"\tUser\u0006;\u0000F[\u0006i\u0002�-I""$2a$10$gEd/xhGMcQTRjtaGzH417e\u0006;\u0000TI"\ranon_key\u0006;\u0000FI"\u001dZmFyemFkQGxhZ29hLmNvbQ==\u0006;\u0000FI"\nflash\u0006;\u0000To:%ActionDispatch::Flash::FlashHash\t:\n@usedo:\bSet\u0006:\n@hash{\u0000:\f@closedF:\r@flashes{\u0006:\u000bnoticeI"\u001cSigned in successfully.\u0006;\u0000T:\t@now0'
Stack trace:
MarshalError: unsupported typecode 189 (index: 176, hex: bd, utf8: �)
Feb 6 15:17:25 scene-composer-app_staging logger: at Marshal._parse (/app/node_modules/marshal/lib/marshal.js:46:15)
Feb 6 15:17:25 scene-composer-app_staging logger: at Marshal._parseArray (/app/node_modules/marshal/lib/marshal.js:124:22)
Feb 6 15:17:25 scene-composer-app_staging logger: at Marshal._parse (/app/node_modules/marshal/lib/marshal.js:37:21)
Feb 6 15:17:25 scene-composer-app_staging logger: at Marshal._parseHash (/app/node_modules/marshal/lib/marshal.js:161:22)
Feb 6 15:17:25 scene-composer-app_staging logger: at Marshal._parse (/app/node_modules/marshal/lib/marshal.js:41:21)
Feb 6 15:17:25 scene-composer-app_staging logger: at Marshal.load (/app/node_modules/marshal/lib/marshal.js:254:26)
Feb 6 15:17:25 scene-composer-app_staging logger: at new Marshal (/app/node_modules/marshal/lib/marshal.js:9:33)
I was using this library as a basis for a similar library in Haskell and noticed along the way that node-marshal doesn't correctly parse values between -129 and -256. I've created a branch with failing test cases here https://github.com/filib/node-marshal/tree/add-failing-test-case.
t.test('-129', function (t) {
t.equals(m.load(intyNegOneTwoNine).parsed, -129, 'should equal -129');
t.end();
});
t.test('-256', function (t) {
t.equals(m.load(intyNegTwoFiveSix).parsed, -256, 'should equal -256');
t.end();
});
[link]
calling this on a MySQL blob:
new Marshal(data.blob.toString(), 'binary')
I get this error for some of the records:
RangeError [ERR_OUT_OF_RANGE]: The value of "offset" is out of range. It must be >= 0 and <= 276. Received 277
at new NodeError (node:internal/errors:371:5)
at boundsError (node:internal/buffer:86:9)
at Buffer.readUInt8 (node:internal/buffer:252:5)
at Marshal._parse (/.../node_modules/marshal/lib/marshal.js:19:32)
at Marshal._parseHash (/.../node_modules/marshal/lib/marshal.js:247:20)
at Marshal._parse (/.../node_modules/marshal/lib/marshal.js:45:21)
at Marshal.load (/.../node_modules/marshal/lib/marshal.js:345:26)
at new Marshal (/.../node_modules/marshal/lib/marshal.js:13:33)
... {
code: 'ERR_OUT_OF_RANGE'
}
The original ruby object is a Hashmap, is this package limited to a certain size of hash?
Unfortunately I cannot provide reproducible data at the time as it contains real user data.
I will try to generate random Hashmaps later to see if I can reproduce this and give you data.
EDIT:
A bit more info
node version: v16.13.2
the MySQL OCTET_LENGTH
gives 277
for the record that generated this particular error.
If I have the following code:
p = Base64.encode64(Marshal.dump(:a))
p = Base64.encode64(Marshal.dump("a"))
I get:
"BAg6BmE=\n"
"BAhJIgZhBjoGRVQ=\n"
If I try to parse these with marshal:
var _symbol = (new Marshal("BAg6BmE=\n",'base64')).parsed
var _string = (new Marshal("BAhJIgZhBjoGRVQ=\n",'base64')).parsed
console.log(_symbol == _string)
I get: True
Is there any way to extend the Marshal parsing algorithm to parse symbols as JavaScript symbols instead?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.