microsoft / napajs Goto Github PK
View Code? Open in Web Editor NEWNapa.js: a multi-threaded JavaScript runtime
License: Other
Napa.js: a multi-threaded JavaScript runtime
License: Other
The following examples can build on Windows and Linux, but fail on macOS
Error message:
SOLINK(target) Release/../../bin/addon.napa
Undefined symbols for architecture x86_64:
"v8::HandleScope::HandleScope(v8::Isolate*)", referenced from:
void napa::module::SetMethod<v8::Localv8::Object >(v8::Localv8::Object const&, char const*, void (*)(v8::FunctionCallbackInfov8::Value const&)) in addon.o
"v8::HandleScope::~HandleScope()", referenced from:
void napa::module::SetMethod<v8::Localv8::Object >(v8::Localv8::Object const&, char const*, void (*)(v8::FunctionCallbackInfov8::Value const&)) in addon.o
"v8::FunctionTemplate::GetFunction()", referenced from:
void napa::module::SetMethod<v8::Localv8::Object >(v8::Localv8::Object const&, char const*, void (*)(v8::FunctionCallbackInfov8::Value const&)) in addon.o
"v8::FunctionTemplate::New(v8::Isolate*, void ()(v8::FunctionCallbackInfov8::Value const&), v8::Localv8::Value, v8::Localv8::Signature, int, v8::ConstructorBehavior)", referenced from:
void napa::module::SetMethod<v8::Localv8::Object >(v8::Localv8::Object const&, char const, void (*)(v8::FunctionCallbackInfov8::Value const&)) in addon.o
"v8::Object::Set(v8::Localv8::Value, v8::Localv8::Value)", referenced from:
void napa::module::SetMethod<v8::Localv8::Object >(v8::Localv8::Object const&, char const*, void (*)(v8::FunctionCallbackInfov8::Value const&)) in addon.o
"v8::String::NewFromUtf8(v8::Isolate*, char const*, v8::String::NewStringType, int)", referenced from:
napa::demo::Method(v8::FunctionCallbackInfov8::Value const&) in addon.o
void napa::module::SetMethod<v8::Localv8::Object >(v8::Localv8::Object const&, char const*, void (*)(v8::FunctionCallbackInfov8::Value const&)) in addon.o
"v8::Isolate::GetCurrent()", referenced from:
void napa::module::SetMethod<v8::Localv8::Object >(v8::Localv8::Object const&, char const*, void (*)(v8::FunctionCallbackInfov8::Value const&)) in addon.o
"v8::Function::SetName(v8::Localv8::String)", referenced from:
void napa::module::SetMethod<v8::Localv8::Object >(v8::Localv8::Object const&, char const*, void (*)(v8::FunctionCallbackInfov8::Value const&)) in addon.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Release/../../bin/addon.napa] Error 1
....cmake-js\node-x64\v8.3.0\include\node\v8.h(4150): warning C4251: 'v8::WasmModuleObjectBuilder::received_buffers_': class 'std::vector<v8::WasmModuleObjectBuilder::Buffer,std::allocator<_Ty>>' needs to have dll-interface to be used by clients of class 'v8::WasmModuleObjectBuilder'
I try to set up CI integration on bitbucket pipelines, Project compiles fine on my Machine (Windows 10)
I am a c++ noop but I feel there is an older c++ compiler version installes on bitbucket? Does anyone know how to work around this problem?
Error: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /opt/atlassian/pipelines/agent/build/node_modules/napajs/bin/napa-binding.node)
at Object.Module._extensions..node (module.js:664:18)
at Module.load (module.js:554:32)
at tryModuleLoad (module.js:497:12)
at Function.Module._load (module.js:489:3)
at Module.require (module.js:579:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/opt/atlassian/pipelines/agent/build/node_modules/napajs/lib/binding.js:22:22)
at Module._compile (module.js:635:30)
at Module.replacementCompile (/opt/atlassian/pipelines/agent/build/node_modules/nyc/node_modules/append-transform/index.js:58:13)
at module.exports (/opt/atlassian/pipelines/agent/build/node_modules/nyc/node_modules/default-require-extensions/js.js:8:9)
at Object.<anonymous> (/opt/atlassian/pipelines/agent/build/node_modules/nyc/node_modules/append-transform/index.js:62:4)
at Module.load (module.js:554:32)
at tryModuleLoad (module.js:497:12)
at Function.Module._load (module.js:489:3)
at Module.require (module.js:579:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/opt/atlassian/pipelines/agent/build/node_modules/napajs/lib/log.js:5:15)
at Module._compile (module.js:635:30)
at Module.replacementCompile (/opt/atlassian/pipelines/agent/build/node_modules/nyc/node_modules/append-transform/index.js:58:13)
at module.exports (/opt/atlassian/pipelines/agent/build/node_modules/nyc/node_modules/default-require-extensions/js.js:8:9)
at Object.<anonymous> (/opt/atlassian/pipelines/agent/build/node_modules/nyc/node_modules/append-transform/index.js:62:4)
at Module.load (module.js:554:32)
at tryModuleLoad (module.js:497:12)
at Function.Module._load (module.js:489:3)
at Module.require (module.js:579:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/opt/atlassian/pipelines/agent/build/node_modules/napajs/lib/index.js:5:13)
at Module._compile (module.js:635:30)
at Module.replacementCompile (/opt/atlassian/pipelines/agent/build/node_modules/nyc/node_modules/append-transform/index.js:58:13)
at module.exports (/opt/atlassian/pipelines/agent/build/node_modules/nyc/node_modules/default-require-extensions/js.js:8:9)
at Object.<anonymous> (/opt/atlassian/pipelines/agent/build/node_modules/nyc/node_modules/append-transform/index.js:62:4)
at Module.load (module.js:554:32)
at tryModuleLoad (module.js:497:12)
at Function.Module._load (module.js:489:3)
at Module.require (module.js:579:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/opt/atlassian/pipelines/agent/build/test/metabolic-device-2-data-decoder-thread.spec.js:6:12)
at Module._compile (module.js:635:30)
at Module.replacementCompile (/opt/atlassian/pipelines/agent/build/node_modules/nyc/node_modules/append-transform/index.js:58:13)
at module.exports (/opt/atlassian/pipelines/agent/build/node_modules/nyc/node_modules/default-require-extensions/js.js:8:9)
at Object.<anonymous> (/opt/atlassian/pipelines/agent/build/node_modules/nyc/node_modules/append-transform/index.js:62:4)
at Module.load (module.js:554:32)
at tryModuleLoad (module.js:497:12)
at Function.Module._load (module.js:489:3)
at Module.require (module.js:579:17)
at require (internal/module.js:11:18)
at /opt/atlassian/pipelines/agent/build/node_modules/mocha/lib/mocha.js:231:27
at Array.forEach (<anonymous>)
at Mocha.loadFiles (/opt/atlassian/pipelines/agent/build/node_modules/mocha/lib/mocha.js:228:14)
at Mocha.run (/opt/atlassian/pipelines/agent/build/node_modules/mocha/lib/mocha.js:514:10)
at Object.<anonymous> (/opt/atlassian/pipelines/agent/build/node_modules/mocha/bin/_mocha:480:18)
at Module._compile (module.js:635:30)
at Module.replacementCompile (/opt/atlassian/pipelines/agent/build/node_modules/nyc/node_modules/append-transform/index.js:58:13)
at module.exports (/opt/atlassian/pipelines/agent/build/node_modules/nyc/node_modules/default-require-extensions/js.js:8:9)
at Object.<anonymous> (/opt/atlassian/pipelines/agent/build/node_modules/nyc/node_modules/append-transform/index.js:62:4)
at Module.load (module.js:554:32)
at tryModuleLoad (module.js:497:12)
at Function.Module._load (module.js:489:3)
at Function.Module.runMain (module.js:676:10)
at runMain (/root/.node-spawn-wrap-91-c99d2e9be88d/node:68:10)
at Function.<anonymous> (/root/.node-spawn-wrap-91-c99d2e9be88d/node:171:5)
at Object.<anonymous> (/opt/atlassian/pipelines/agent/build/node_modules/nyc/bin/wrap.js:23:4)
at Module._compile (module.js:635:30)
at Object.Module._extensions..js (module.js:646:10)
at Module.load (module.js:554:32)
at tryModuleLoad (module.js:497:12)
at Function.Module._load (module.js:489:3)
at Function.Module.runMain (module.js:676:10)
at /root/.node-spawn-wrap-91-c99d2e9be88d/node:178:8
at Object.<anonymous> (/root/.node-spawn-wrap-91-c99d2e9be88d/node:181:3)
at Module._compile (module.js:635:30)
at Object.Module._extensions..js (module.js:646:10)
at Module.load (module.js:554:32)
at tryModuleLoad (module.js:497:12)
at Function.Module._load (module.js:489:3)
at Function.Module.runMain (module.js:676:10)
at startup (bootstrap_node.js:187:16)
at bootstrap_node.js:608:3
follow the steps in estimate-pi-in-parallel
testes in ubuntu 16.04
$ node estimate-pi-in-parallel.js
module.js:653
return process.dlopen(module, path._makeLong(filename));
^
Error: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.22' not found (required by /home/tigercosmos/Desktop/napajs/examples/tutorial/estimate-pi-in-parallel/node_modules/napajs/bin/libnapa.so)
at Object.Module._extensions..node (module.js:653:18)
at Module.load (module.js:545:32)
at tryModuleLoad (module.js:508:12)
at Function.Module._load (module.js:500:3)
at Module.require (module.js:568:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/home/tigercosmos/Desktop/napajs/examples/tutorial/estimate-pi-in-parallel/node_modules/napajs/lib/binding.js:5:22)
at Module._compile (module.js:624:30)
at Object.Module._extensions..js (module.js:635:10)
at Module.load (module.js:545:32)
If you change the Fibonacci example to:
// Run fibonacci evaluation in sequence.
run(20)
Sometime you receive:
Fatal error in ../deps/v8/src/runtime/runtime-promise.cc, line 59
Check failed: args[0]->IsJSPromise().
Illegal instruction
Sometimes a segmentation fault:
Nth Fibonacci # of workers latency in MS
-----------------------------------------------------------
run n: 20
Segmentation fault
This happen running with debian 9.2
It does not work because contextDir
in ModuleLoader::ModuleLoaderImpl::RequireModule
is set to node_modules\napajs\lib\zone
instead of the working directory.
https://travis-ci.com/Microsoft/napajs/jobs/83906089
Error: dlopen(/Users/travis/build/Microsoft/napajs/bin/napa-binding.node, 1): Symbol not found: __ZTIN2v811ArrayBuffer9AllocatorE
I try to run code in https://github.com/Microsoft/napajs/blob/master/examples/tutorial/recursive-fibonacci/recursive-fibonacci.js. When is change to run run(20)
, it has errors in below:
node(10345,0x7fffb809e3c0) malloc: *** error for object 0x103417b50: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
[1] 10345 abort node napa.js
Napajs does not work with Nodejs of 8.8.1+ version.
I tried to rebuild napajs with "npm rebuild napajs --force" but it has no effect.
I, this is my dev env :
Node Version: 7.10.0
Npm Version: 4.2.0
OS: UBUNTU
A simple require of napajs generates this error :
Error: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version 'CXXABI_1.3.9' not found
Is it possible to "force" napa.js to use the version of the os?
With a variable environment?
Or another solution?
Thank you in advance
The major motivation is to let users discover new features, improvements and bug fixes in each versions in a human readable manner.
We can follow Node's practice:
https://github.com/nodejs/node/blob/master/CHANGELOG.md
Now there is no way to get that of node isolate
hello,
do you have any examples of implementation of the module fs.readFileSync with napajs?
// Mocha test.
var napa = require('napajs');
describe('a-test', function() {
var zone = napa.zone.create('zone1');
it('should succeed', function () {
return zone.broadcast("var state = 0;");
}).timeout(1000)
}
Test should pass.
Hang until timeout has been reached.
console.log(require.resolve('path-not-exist'));
Exception "Error: Cannot find module 'path-not-exist'"
undefined
Inconsistent behavior with Node.js.
Hello
I am very new to use Napa module and just implemented a hello world web server using Napa module. This web server just prints "Hello Napa" in parallel by multiple threads. However, the problem is this web server cannot respond to client. I am not sure if the reason is Napa module does not support node express or http module or else. I know Napa has two zones, one is napa zone and the other is node zone, if napa does not support http or express module, how can I use node zone to compensate that? Thanks!
I put my code below,
var napa = require('napajs');
var app = require('express')();
var Promise = require('promise');
const NUMBER_OF_WORKERS = 4
, PORT = 3005
, HOST = '0.0.0.0';
var zone = napa.zone.create('zone', { workers: NUMBER_OF_WORKERS });
app.listen(PORT, HOST);
function startParellelRun(){
app.get('/', (req, res) => {
run().then(() => {
res.status(200).send('OK');
}).catch(() => {
res.status(500).send('ERROR');
});
});
}
function run(){
var promises = [];
for(var i = 0; i < NUMBER_OF_WORKERS; i++){
promises[i] = zone.execute(hello, [i]);
}
return Promise.all(promises)
.then((values) => {
values.forEach((ele) => {
console.log(ele.value);
});
});
}
function hello(i){
return 'Hello Napa! I am Worker ' + i;
}
startParellelRun();
Napa.js doesn't have fully support on Windows UNC prefix \\?\
in filename.
path.join()
behaviors incorrectly in Napa.js when using UNC prefixpath.resolve()
behaviors incorrectly in Napa.js when using UNC prefixpath.toNamespacedPath()
(path._makeLong()
in node< 9.0)path.toNamespacedPath()
)console.time('timeout test');
zone.execute(() => { while (true) {} }, [], { timeout: 50})
.catch((error) => {
console.timeEnd('timeout test');
});
timeout test: varies between 50 - 60ms
timeout test: around 50.00 ms
The problem is raised from sorting big array in parallel
This is a common pattern to share big payload across workers. We also had it tracked in the roadmap.
Hi guys, wouldn't it be something cool to get a .deb or .rpm package available?
It would be easier for those willing to went native with NapaJS instead of using it as a NodeJS module.
Plus, this would also accelerate a widespread adoption.
Getting this error when trying to run a script that uses napajs. Tested using recursive-fibonacci
as well.
colton is in ~/Code/napajs/examples/tutorial/recursive-fibonacci on (branch:master)
» node recursive-fibonacci.js
module.js:598
return process.dlopen(module, path._makeLong(filename));
^
Error: dlopen(/Users/colton/node_modules/napajs/bin/napa-binding.node, 1): Library not loaded: @rpath/libnapa.dylib
Referenced from: /Users/colton/node_modules/napajs/bin/napa-binding.node
Reason: image not found
at Object.Module._extensions..node (module.js:598:18)
at Module.load (module.js:503:32)
at tryModuleLoad (module.js:466:12)
at Function.Module._load (module.js:458:3)
at Module.require (module.js:513:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/Users/colton/node_modules/napajs/lib/binding.js:5:22)
at Module._compile (module.js:569:30)
at Object.Module._extensions..js (module.js:580:10)
at Module.load (module.js:503:32)
function test() {
var m = require('some-module');
}
Only performance hit from require.resolve on first require, afterwards it would be very fast.
We only have module cache on canonical path, every require
will do require.resolve
, which involves a lot of IO on file lookup.
var napa = require('napajs');
var node = napa.zone.node;
function test(n) {
if (n == 2)
return;
console.log("Forward trace", n);
return node.execute(test, [n + 1])
.then(() => console.log('Back trace', n));
}
node.execute(test, [0])
.catch(err => console.log(err))
.then(() => { console.log('Finished'); });
Forward trace 0
Forward trace 1
Back trace 1
Back trace 0
Finished
Forward trace 0
Forward trace 1
<program hang>
Napa zone with 1 worker works as expected.
Sample code:
const napa = require("napajs");
const fs = require("fs");
const myUTFString="éóíúã’“";
const zone = napa.zone.create("zone", {workers: 1});
const store = napa.store.create("store");
store.set("string", myUTFString);
zone.broadcast(`
function example () {
const str = global.napa.store.get("store").get("string");
console.log(str);
return str;
}
`);
zone.execute("", "example").then(result => {
console.log(result.value)
});
Expected output:
éóíúã’“
éóíúã’“
Actual output:
éóÃúãâ��â��
éóÃúãâ��â��
This is on Windows 10.
Change e0696c5 introduced this bug.
When zone.execute('path', ...) , the first arg will be rewrited to join with the current folder, thus cause it not recognized as built-in module.
let napaZone = napa.zone.create('test-dirname-zone', { workers: 1 });
let nodeZone = napa.zone.node;
napaZone.broadcast('var getDirname = function () { return __dirname; };');
nodeZone.broadcast('var getDirname = function () { return __dirname; };');
napaZone.execute("", "getDirname")
This returns the working directory.
nodeZone.execute("", "getDirname")
This throws exception saying that '__dirname' is undefined.
Say, how can I broadcast such code in workers?
const f = require('../lib/my_function')
...
https://travis-ci.com/Microsoft/napajs/jobs/83906092
napajs/module
load
*** Error in `/home/travis/.nvm/versions/node/v4.8.4/bin/node': free(): invalid pointer: 0x00007efecb431b80 ***
Aborted (core dumped)
npm ERR! Test failed. See above for more details.
Now npm search
will show snippet from README.md, which is
[![Build Status for Linux/MacOS](https://travis-ci.org/Microsoft/napajs.svg?branch=master)](https://travis-ci.org/Microsoft/napajs) [![Build Status for Windows](https://ci.appveyor.com/api/projects/status/github/Microsoft/napajs?branch=master&svg=true)](
Adding "description" field in package.json will solve this.
Pre-built binaries are not working on Linux and MacOS.
npm install napajs
node -e "require('napajs')"
module.js:597
return process.dlopen(module, path._makeLong(filename));
^
Error: libnapa.so: cannot open shared object file: No such file or directory
Update napa.js to version 0.1.4 or later to resolve this issue.
1. export LD_LIBRARY_PATH=./node_modules/napajs/bin:$LD_LIBRARY_PATH
before launch node.
2. Use npm install --no-fetch
so the install script will fallback to build napajs. See this for more information.
The file 'napa-binding.node' was using absolute path in its rpath before v0.1.4. This causes the pre-built file cannot be loaded in runtime.
header file napa/module.h uses #pragma to disable warning C4100 when including v8.h.
napa.h contains napa/module.h, however it fails to disable that warning because v8.h is included before napa/module.h
Current workaround:
#include "napa.h"
// this produces the warning
change to
#include "napa/module.h"
#include "napa.h"
// no warning now.
var napa = require('napajs');
var z1 = napa.zone.create('z1');
z1.broadcast("var co = require('co');"); // crash in this line
Can I call zone.execute(name, async () => ...)
? Does it works as expected?
At the moment the docs show the following table in https://github.com/Microsoft/napajs/tree/master/examples/tutorial/estimate-pi-in-parallel
# of points # of batches # of workers latency in MS estimated π deviation
---------------------------------------------------------------------------------------
4000000 1 4 218 3.141958 0.0003653464
4000000 2 4 110 3.141953 0.0003603464
4000000 4 4 78 3.139600 0.001992654
4000000 8 4 62 3.142732 0.001139346
I would expect that the number of batches would not effect the accuracy of the estimate.
But it looks like there is a big dependence on it - which I'm guessing is due to reusing the same
RNG seeds or something like that.
Either way the source of that issue should be explained.
Running Node 6.11.2 and getting the following error
module.js:597
return process.dlopen(module, path._makeLong(filename));
^
Error: dlopen(/Users/rashid/Yellow-Messenger/node-threded/node_modules/napajs/bin/napa-binding.node, 1): Library not loaded: @rpath/libnapa.dylib
Referenced from: /Users/rashid/Yellow-Messenger/node-threded/node_modules/napajs/bin/napa-binding.node
Reason: image not found
at Error (native)
at Object.Module._extensions..node (module.js:597:18)
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)
at Object.<anonymous> (/Users/rashid/Yellow-Messenger/node-threded/node_modules/napajs/lib/binding.js:5:22)
at Module._compile (module.js:570:32)
at Object.Module._extensions..js (module.js:579:10)```
var napa = require('napajs');
var zone = napa.zone.create('zone1');
zone.broadcast('var a = 0');
zone.execute(() => { console.log(a); }, []);
0
ReferenceError: a is not defined.
Any future plans for embedding ts-node
or straight-up TS support?
( ͡° ͜ʖ ͡°)
(You knew this issue was going to happen at some point.)
That or you know, there's an NPM module called clustering out there. That's all this really is, glorified clustering. Even though I had better luck with symbols in clustering. I would switch to something like C.
I install napa by npm install napa
, then run the demo beblow
var napa = require('napajs');
var zone1 = napa.zone.create('zone1', { workers: 4 });
// Broadcast code to all 4 workers in 'zone1'.
zone1.broadcast('console.log("hello world");');
// Execute an anonymous function in any worker thread in 'zone1'.
zone1.execute(
(text) => text,
['hello napa'])
.then((result) => {
console.log(result.value);
});
I get this error
Error: dlopen(/Xxx/testplace/napa/node_modules/.0.1.3@napajs/bin/napa-binding.node, 1): Library not loaded: @rpath/libnapa.dylib
Referenced from: /Xxx/testplace/napa/node_modules/.0.1.3@napajs/bin/napa-binding.node
Reason: image not found
how to solve it?
// file path: /usr/a/test.js
zone.execute(() => { console.log(__dirname); });
/usr/a/test.js
<napajs-install-root>/lib/transport
This behavior is causing confusion when calling require
in anonymous function in execute
.
zone.js by Angular
Zones spec proposal for TC39
Zones from Dart lang
It might be a good idea to mention that the term zone used here is different from what JavaScript developers might have heard of in the past (from Angular which created zone.js)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.