clayzermk1 / node-marshal Goto Github PK

Hi,
when I want to try deserialize a rails 2.3 session data with some customized key inside the data.
var sdata = new Buffer(session_data, 'base64');
var m = new Marshal(sdata);

I got
MarshalError: unsupported typecode 108 (index: 57, hex: 6c, utf8: l)

After further investigation in ruby 1.8 marshal.c, seems that l is for 'long' (?) which I think it's the equivalent of bignum in ruby. And I know the data inside is integer (Time.now.to_i), I don't why ruby/rails treat it as bignum when serializing it. If you can add support for bignum that would be great.

Hi there, thanks for this useful library!

It seems that there might be a problem with parsing Bignums, specifically in the context of a hash. For reasons I don't understand, I can't actually make a test where it fails - but I can make it fail just fine in the REPL. Consider the following:

2.2.4 :045 >   Marshal.dump({1=>1499769350, 2=>2}).unpack('H*')
 => ["04087b0769066c2b0706aa645969076907"] 

> var Marshal = require('marshal')
undefined
> var m = new Marshal
undefined
> m.load(new Buffer("04087b0769066c2b0706aa645969076907", 'hex')).parsed
MarshalError: unsupported typecode 43 (index: 7, hex: 2b, utf8: +)
    at Marshal._parse (/home/dvangeest/projects/test/node_modules/marshal/lib/marshal.js:52:15)
    at Marshal._parseHash (/home/dvangeest/projects/test/node_modules/marshal/lib/marshal.js:170:20)
    at Marshal._parse (/home/dvangeest/projects/test/node_modules/marshal/lib/marshal.js:45:21)
    at Marshal.load (/home/dvangeest/projects/test/node_modules/marshal/lib/marshal.js:264:26)
    at repl:1:3
    at sigintHandlersWrap (vm.js:22:35)
    at sigintHandlersWrap (vm.js:96:12)
    at ContextifyScript.Script.runInThisContext (vm.js:21:12)
    at REPLServer.defaultEval (repl.js:346:29)
    at bound (domain.js:280:14)

I tried unsuccessfully to add a unit test for this case... it passes, which makes me wonder if a) I'm just doing something wrong, or b) if there's a problem with the test framework.

Based on my limited understanding of Ruby's Marshal format, it looks like the index is not being advanced beyond the typecode for Bignum. Therefore, the + following the l, denoting that it's a positive number, is read as the next type code. My JS is not good enough to spot the error, if indeed there is one.

Any ideas?

An error occurred when the } symbol was parsed

my js code

const fs = require('fs')
const Marshal = require('marshal');

const buf = fs.readFileSync('./Save3.rxdata', {
    encoding: 'hex'
})
var m = new Marshal(buf, 'hex');
console.log(m.parsed);

Save3.rxdata:

👋 this endpoint /api/v1/dependencies states:

Returns a marshalled array of hashes for all versions of given gems. Each hash contains a gem version with its dependencies making this useful for resolving dependencies.

I'm buffering the response into a Buffer and then I'm passing it to this module as new Marshal(buf)

However, it throws an error:

MarshalError: unsupported typecode 8 (index: 2, hex: 08, utf8:)
    at Marshal._parse (/node_modules/marshal/lib/marshal.js:54:15)
    at Marshal.load (/node_modules/marshal/lib/marshal.js:345:26)
    at new Marshal (/node_modules/marshal/lib/marshal.js:13:33)

I can provide you with a simple snippet that reproduces the issue.

The following string throws an exception at parse time:

'\u0004\b{\nI"\u000fsession_id\u0006:\u0006ETI"%621c43c4db9732740e2fafd941635f0a\u0006;\u0000TI"\u0010_csrf_token\u0006;\u0000FI"1REC7hnjqnd0mNgEzS0+dYTEzUY7NIL1WbAjMiiHv5kc=\u0006;\u0000FI"\u0019warden.user.user.key\u0006;\u0000T[\bI"\tUser\u0006;\u0000F[\u0006i\u0002�-I""$2a$10$gEd/xhGMcQTRjtaGzH417e\u0006;\u0000TI"\ranon_key\u0006;\u0000FI"\u001dZmFyemFkQGxhZ29hLmNvbQ==\u0006;\u0000FI"\nflash\u0006;\u0000To:%ActionDispatch::Flash::FlashHash\t:\n@usedo:\bSet\u0006:\n@hash{\u0000:\f@closedF:\r@flashes{\u0006:\u000bnoticeI"\u001cSigned in successfully.\u0006;\u0000T:\t@now0'

Stack trace:
MarshalError: unsupported typecode 189 (index: 176, hex: bd, utf8: �)
Feb 6 15:17:25 scene-composer-app_staging logger: at Marshal._parse (/app/node_modules/marshal/lib/marshal.js:46:15)
Feb 6 15:17:25 scene-composer-app_staging logger: at Marshal._parseArray (/app/node_modules/marshal/lib/marshal.js:124:22)
Feb 6 15:17:25 scene-composer-app_staging logger: at Marshal._parse (/app/node_modules/marshal/lib/marshal.js:37:21)
Feb 6 15:17:25 scene-composer-app_staging logger: at Marshal._parseHash (/app/node_modules/marshal/lib/marshal.js:161:22)
Feb 6 15:17:25 scene-composer-app_staging logger: at Marshal._parse (/app/node_modules/marshal/lib/marshal.js:41:21)
Feb 6 15:17:25 scene-composer-app_staging logger: at Marshal.load (/app/node_modules/marshal/lib/marshal.js:254:26)
Feb 6 15:17:25 scene-composer-app_staging logger: at new Marshal (/app/node_modules/marshal/lib/marshal.js:9:33)

I was using this library as a basis for a similar library in Haskell and noticed along the way that node-marshal doesn't correctly parse values between -129 and -256. I've created a branch with failing test cases here https://github.com/filib/node-marshal/tree/add-failing-test-case.

t.test('-129', function (t) {
  t.equals(m.load(intyNegOneTwoNine).parsed, -129, 'should equal -129');
  t.end();
});
t.test('-256', function (t) {
  t.equals(m.load(intyNegTwoFiveSix).parsed, -256, 'should equal -256');
  t.end();
});

[link]

calling this on a MySQL blob:

new Marshal(data.blob.toString(), 'binary')

I get this error for some of the records:

RangeError [ERR_OUT_OF_RANGE]: The value of "offset" is out of range. It must be >= 0 and <= 276. Received 277
    at new NodeError (node:internal/errors:371:5)
    at boundsError (node:internal/buffer:86:9)
    at Buffer.readUInt8 (node:internal/buffer:252:5)
    at Marshal._parse (/.../node_modules/marshal/lib/marshal.js:19:32)
    at Marshal._parseHash (/.../node_modules/marshal/lib/marshal.js:247:20)
    at Marshal._parse (/.../node_modules/marshal/lib/marshal.js:45:21)
    at Marshal.load (/.../node_modules/marshal/lib/marshal.js:345:26)
    at new Marshal (/.../node_modules/marshal/lib/marshal.js:13:33)
    ... {
  code: 'ERR_OUT_OF_RANGE'
}

The original ruby object is a Hashmap, is this package limited to a certain size of hash?
Unfortunately I cannot provide reproducible data at the time as it contains real user data.
I will try to generate random Hashmaps later to see if I can reproduce this and give you data.

EDIT:
A bit more info
node version: v16.13.2
the MySQL OCTET_LENGTH gives 277 for the record that generated this particular error.

If I have the following code:

p = Base64.encode64(Marshal.dump(:a))
p = Base64.encode64(Marshal.dump("a"))

I get:

"BAg6BmE=\n"
"BAhJIgZhBjoGRVQ=\n"

If I try to parse these with marshal:

var _symbol = (new Marshal("BAg6BmE=\n",'base64')).parsed
var _string = (new Marshal("BAhJIgZhBjoGRVQ=\n",'base64')).parsed
console.log(_symbol == _string)

I get: True

Is there any way to extend the Marshal parsing algorithm to parse symbols as JavaScript symbols instead?