jlaine / node-jpickle Goto Github PK

View Code? Open in Web Editor NEW

45.0 45.0 13.0 44 KB

Full-javascript parser for Python's pickle format

License: MIT License

JavaScript 87.37% Python 12.63%

node-jpickle's People

Contributors

Stargazers

Watchers

Forkers

epowers geiseri romanalexander drc0 shaunc guoyu07 hakanaku1234 amrlotfy77 ch-hristov leai-jiang foxt badideaexception jorgesardina

node-jpickle's Issues

Example code raises `Uncaught ReferenceError: Buffer is not defined`

Got this error while trying the example code from the README:

Uncaught ReferenceError: Buffer is not defined
    load webpack:///./node_modules/jpickle/lib/jpickle.js?:82
    loads webpack:///./node_modules/jpickle/lib/jpickle.js?:352
    <anonymous> webpack:///./src/index.js?:3
    js http://localhost:8080/bundle.js:29
    __webpack_require__ http://localhost:8080/bundle.js:358
    <anonymous> http://localhost:8080/bundle.js:1426
    <anonymous> http://localhost:8080/bundle.js:1428

This is my very first attempt at doing anything related to frontend, so I'm dumping everything here hoping that it's useful for debugging.

$ tree -I node_modules
.
├── dist
│   ├── bundle.js
│   └── index.html
├── package.json
├── package-lock.json
├── src
│   ├── index.html
│   └── index.js
└── webpack.config.js

2 directories, 7 files
$ cat src/index.html 
<html>
    <head>
        <meta charset="utf-8">
        <title>Test</title>
    </head>
    <body>
        <h1>Hello, world!</h1>
    </body>
</html>
$ cat src/index.js
var jpickle = require('jpickle');
var contents = jpickle.loads('U\x0bhello worldq\x01.');
console.log(contents);

$ cat webpack.config.js 
const path = require('path');
const HtmlWebpackPlugin = require('html-webpack-plugin');

module.exports = {
  //This property defines where the application starts
  entry:'./src/index.js',
    
  //This property defines the file path and the file name which will be used for deploying the bundled file
  output:{
    path: path.join(__dirname, '/dist'),
    filename: 'bundle.js'
  },
    
  //Setup loaders
  module: {
    rules: [
      {
        test: /\.js$/, 
        exclude: /node_modules/,
        use: {
          loader: 'babel-loader'
        }
      }
    ]
  },
    
  // Setup plugin to use a HTML file for serving bundled js files
  plugins: [
    new HtmlWebpackPlugin({
      template: './src/index.html'
    })
  ]
}

$ cat package.json 
{
  "dependencies": {
    "jpickle": "github:jlaine/node-jpickle",
    "webpack": "^5.65.0",
    "webpack-cli": "^4.9.1",
    "webpack-dev-server": "^4.6.0"
  },
  "scripts": {
    "start": "webpack-dev-server --mode development --open --hot",
    "build": "webpack --mode production"
  },
  "devDependencies": {
    "babel-core": "^6.26.3",
    "babel-loader": "^8.2.3",
    "html-webpack-plugin": "^5.5.0"
  }
}

Add support for

Hi,
Will you consider adding support for long binput? i came across this working with graphite pickle interface

I suggest this fix:
add:
LONG_BINPUT = 'r'

and add:
case LONG_BINPUT:
var index = buffer.readUInt32LE(i);
i+=4;
this.memo['' + index] = this.stack[this.stack.length-1];
break;

Thanks in advance

Shay.

Why does proto not return the proto read

Why does proto not return the proto read , eg. 2, or 3 ?

"Unhandled opcode ''"

I have an error on load

"Unhandled opcode '�'"

This is my nodejs code

let pickled = fs.readFileSync('./py-sample/pickle', {
      encoding: 'latin1',
    });
try {
      jpickle.emulated['src.sample.Sample'] = MyClass;
      var unpickled = jpickle.loads(pickled.toString());
      return { msg: unpickled };
  } catch (e) {
    return { e, err: pickled.toString() };
  }

This is the python sources

the class serialized (in directory src/sample.py)

class Sample():
    def __init__(self):
        self.un = 1
        self.deux = 2
        self.trois = 3


    def sum(self):
        return "{}".format(self.un + self.deux + self.trois)

the serialization code

import pickle
from src.sample import Sample
sample = Sample()
with open('pickle', 'wb') as target:
    pickle.dump(sample, target, protocol=2)

as you can see I force the use of the protocole 2

here the generated pickle file

��csrc.sample
Sample
q�)�q�}q�(X����unq�K�X����deuxq�K�X����troisq�K�ub.

Do you have already see the problem and could you explain where is/are my mistakes ?

I'd like to convert data from here: https://github.com/lmmentel/mendeleevapp/tree/master/app/data into JSON. I checked the code, it does new Buffer(param, "binary") with the param, which is deprecated if the param is a bufffer. I tried to load the file several ways and got different error messages each time:

jpickle.loads(fs.readFileSync("./neutral_0.3.2.pkl"));
Unhandled opcode '128'

jpickle.loads(fs.readFileSync("./neutral_0.3.2.pkl", "utf8"));
Unhandled opcode '�'

jpickle.loads(fs.readFileSync("./neutral_0.3.2.pkl", "binary"));
Unhandled pickle protocol version: 4

Any idea why it cannot parse the file?

does it recreate a new object?

does it make a snapshot of an object like python using pickle, or recreate a new object?

Fail to load nested array in pickle version 2 format

There is a typo in jpickle.js line 49:

  , LONG_BINGET = 'h'

should be:

  , LONG_BINGET = 'j'

to generate a test file:

import pickle
a = [('a.b.c', (1405960348.0, 999.0)),
('a.b.c', (1405960348.0, 999.0))]

filename = open('out.bin', 'wb')
pickle.dump(a,filename,protocol=-1);
filename.close()

I'd like to emulate Numpy arrays, but dont know where to begin

This pickle program is amazing offers a very straightforward way to pass large python objects across to Javascript as a binary transfer, rather than converting to Json or something equally inefficient.

But I am stuck on one point: numpy arrays.

They get me to Case: GLOBAL , module = numpy.core.multiarray which wants to drop into an emulated class but I'm not sure how to proceed further.

Has anyone else worked on this? Thanks

how can i import this package on typescript project ?

var jpickle = require('jpickle');
jpickle.loads('U\x0bhello worldq\x01.');

Unhandled opcode '128'

Hi, I'm trying to decode Pickle'd objects in JS.

Running the following code works

pickle.loads(open("test2.bin","rb").read(),encoding='latin1')

However this does not

var f = Pickle.loads(AUint8Array)

I get ` Unhandled opcode '128' (the file starts with 80 02 7D 71 01 28 58 17 00 00 00)

Any help would be appreciated.

jlaine / node-jpickle Goto Github PK

node-jpickle's People

Contributors

Stargazers

Watchers

Forkers

node-jpickle's Issues

Example code raises `Uncaught ReferenceError: Buffer is not defined`

Add support for

Why does proto not return the proto read

"Unhandled opcode ''"

unhandled opcode

does it recreate a new object?

Fail to load nested array in pickle version 2 format

I'd like to emulate Numpy arrays, but dont know where to begin

how can i import this package on typescript project ?

Unhandled opcode '128'

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent