namgk / dnr-editor Goto Github PK
View Code? Open in Web Editor NEWDistributed Data-Flow Coordination Platform Based on Node-RED
License: Apache License 2.0
Distributed Data-Flow Coordination Platform Based on Node-RED
License: Apache License 2.0
Currently now when you export a flow on the UI, link constraints are not exported yet.
I have got error in front-end when trying import dnr-daemon node to node-red:
"TypeError: Cannot read property 'deviceId' of null"
Text from node-red console:
[error] [dnr-daemon:1888cbc9.fb96b4] TypeError: Cannot read property 'deviceId' of null
Exported data from dnr-editor:
[{"id":"af25fe8d.debf5","type":"tab","label":"DNR Seed"},{"id":"f14195aa.25e298","operatorUrl":"http://127.0.0.1:1818","type":"dnr-daemon","x":100,"y":100,"z":"af25fe8d.debf5"}]
dnr-editor version: v 0.17.3-1.0.2
Node-RED version: v0.18.2
node-red-contrib-dnr 3.5.16
currently new nodes are install fully via npm install. This sometime requires building of the source code, which in turn might not be necessary as we only care about the node's definition.
--> need a way to transform the node source into just node definition to strip out all the source part.
Hi
I'm not sure if this query is related to that of 5 Feb (sorry if so - but the errors I have are different). I am trying to install dnr-editor on a VM using Centos 7. I've added the NodeSource yum repository and installed node.js using sudo yum install -y nodejs. This gives me version 10.15.3 of Node.js. This is all that is installed.
I try to install dnr-editor using 'npm install -g dnr-editor' and it seems to go quite smoothly (ignoring some deprecated files) until it tries to access '/usr/lib/node_modules' when there is a permission denied message - see final lines of the log file.
Any idea - I don't seem to have done enough to go far wrong?
Thanks
dnr_editor_log.txt
Hello Namgk
Thank you for creating this amazing tool. I am completely new to nodejs and node-red so please allow me to ask a very rookie question. I am currently attempting to install the dnr-editor via both npm and git clone as instructed. However I am running into a series of warnings including many deprecated packages. I have attempted to install some of them individually however the problem still persist. Could you please help me take a look at the log and perhaps point out what I might have done wrong. Thanks in advance for your time and attention!
npm dnr editor install output.txt
the project has gone quite large, dnr.js now stink! However, refactoring require changing in main.js to have new files to be concatenated into the final red.js one. That could be a pain.
I am trying to use DNR for the first time, but everytime I deploy the flow below, it requires me to merge the event on each server.
I deployed the Editor on Master Node 1
And have 2 slaves.
So I have just a basic test enviroment
I have all three nodes
/home/eperry/node-red/node-red-dnr/
Master flow and settings port 1818
/home/eperry/node-red/node-red-dnr/flows/[settings|flow.js]
Slave Flow and settings port 1819
/home/eperry/node-red/node-red-dnr/flows-slave1/[settings|flow.js]
Slave Flow and settings port 1820
/home/eperry/node-red/node-red-dnr/flows-slave2/[settings|flow.js]
RPi nodes not loaded, this is because the nodes check if node-red is running on a Pi or not. In this case of Dnr-Editor, we only care about the node's definition.
--> need to "emptify" all the node's constructor so that no code will be run on server.
Hi, I'm a master degree student.
For my thesis work I want to use a different broker in Distributed NodeRed, I saw in this project it is used MQTT.
I think in dnr editor I have to modify the following files to use a new Broker:
Instead in node-red-contrib-dnr I have to modify:
-https://github.com/namgk/node-red-contrib-dnr/blob/master/dnr/dist-wire.js
-https://github.com/namgk/node-red-contrib-dnr/blob/master/dnr/broker.js
-https://github.com/namgk/node-red-contrib-dnr/blob/master/dnr/daemon.js
Please, could you reply me If I'm working in the right way?
Hi,
I'm trying to send an email in a distributed node-red setup.
Unforunately the (send-)email object is also different and failes to send an email with error message "error: self signed certificate", but in my tests I just tried to send out an email using (unsecure) smtp without TLS etc.
It seems that the Node-Red Editor is based on a very old version of Node-Red.
Is there a way to fix this? Is there an easy way to rebase DNR on a more recent Node-Red Version?
Sorry for placing such newby questions...
-Version of DNR-Editor: v.0.19.4-4.1.0 (latest)
-Version of Node-Red v1.0.3
Here is the corresponding JSON of this flow:
[{"id":"ee6711f4.ef043","type":"http in","z":"1b53be18.b0a122","name":"","url":"/test0","method":"get","upload":false,"swaggerDoc":"","x":120,"y":160,"wires":[["2671010.e829e"]],"constraints":{}},{"id":"e680303d.fb92b","type":"http response","z":"1b53be18.b0a122","name":"","statusCode":"","headers":{},"x":470,"y":160,"wires":[],"constraints":{}},{"id":"2671010.e829e","type":"function","z":"1b53be18.b0a122","name":"LogOnly2","func":"msg.payload = {\"a\":\"b\"};\nconsole.log(\"log2\");\nreturn msg;","outputs":1,"noerr":0,"x":300,"y":160,"wires":[["e680303d.fb92b"]],"constraints":{}}]
Here is the corresponding JSON of this flow:
[{"id":"19b6afe.fea9d5","type":"subflow","name":"Subflow 7","info":"","in":[{"x":160,"y":160,"wires":[{"id":"2671010.e829e"}]}],"out":[{"x":440,"y":160,"wires":[{"id":"2671010.e829e","port":0}]}]},{"id":"2671010.e829e","type":"function","z":"19b6afe.fea9d5","name":"LogOnly2","func":"msg.payload = {\"a\":\"b\"};\nconsole.log(\"log2\");\nreturn msg;","outputs":1,"noerr":0,"x":300,"y":160,"wires":[[]],"constraints":{}},{"id":"ee6711f4.ef043","type":"http in","z":"1b53be18.b0a122","name":"","url":"/test0","method":"get","upload":false,"swaggerDoc":"","x":120,"y":160,"wires":[["5de036f6.b93638"]],"constraints":{}},{"id":"e680303d.fb92b","type":"http response","z":"1b53be18.b0a122","name":"","statusCode":"","headers":{},"x":470,"y":160,"wires":[],"constraints":{}},{"id":"5de036f6.b93638","type":"subflow:19b6afe.fea9d5","z":"1b53be18.b0a122","x":300,"y":160,"wires":[["e680303d.fb92b"]]}]
Before "Selection to Subflow", I can visit http://localhost:12002/test0 on browser (without any error on terminal also).
After "Selection to Subflow" and I visit http://localhost:12002/test0 again on browser, it shows a error TypeError: Cannot read property 'configs' of undefined
on Node-RED instance 1 (run on port 12001).
TypeError: Cannot read property 'configs' of undefined
at createSubflow (/Users/weishi/Desktop/OneDrive/Dissertation/Project/NodeRed-subnode2/node-red/red/runtime/nodes/flows/Flow.js:341:18)
at Flow.start (/Users/weishi/Desktop/OneDrive/Dissertation/Project/NodeRed-subnode2/node-red/red/runtime/nodes/flows/Flow.js:97:41)
at start (/Users/weishi/Desktop/OneDrive/Dissertation/Project/NodeRed-subnode2/node-red/red/runtime/nodes/flows/index.js:327:29)
at /Users/weishi/Desktop/OneDrive/Dissertation/Project/NodeRed-subnode2/node-red/red/runtime/nodes/flows/index.js:164:21
at tryCatchReject (/Users/weishi/Desktop/OneDrive/Dissertation/Project/NodeRed-subnode2/node-red/node_modules/when/lib/makePromise.js:845:30)
at runContinuation1 (/Users/weishi/Desktop/OneDrive/Dissertation/Project/NodeRed-subnode2/node-red/node_modules/when/lib/makePromise.js:804:4)
at Fulfilled.when (/Users/weishi/Desktop/OneDrive/Dissertation/Project/NodeRed-subnode2/node-red/node_modules/when/lib/makePromise.js:592:4)
at Pending.run (/Users/weishi/Desktop/OneDrive/Dissertation/Project/NodeRed-subnode2/node-red/node_modules/when/lib/makePromise.js:483:13)
at Scheduler._drain (/Users/weishi/Desktop/OneDrive/Dissertation/Project/NodeRed-subnode2/node-red/node_modules/when/lib/Scheduler.js:62:19)
at Scheduler.drain (/Users/weishi/Desktop/OneDrive/Dissertation/Project/NodeRed-subnode2/node-red/node_modules/when/lib/Scheduler.js:27:9)
Can visit http://localhost:12002/test0 correctly.
It seems that the reason of that is there is no definition of the "Subflow 7" in Node-RED instance 1 (run on port 12001).
Create an output link to others flows
Input links works perfectly.
When I deploy my dnr-editor flow with a output link to others flows, all the flows crash.
I notice that when I do a deploy, the name of the flows in the output links are not being updated to the name of the flows in the node-red cluster.
The input links do this updated by default
I have one instance containing dnr-editor and two instances with node-red integrated to dnr-editor.
On the Node-RED client the following method in Flows.js receives an undefined value for sf, resulting in access violation:
function createSubflow(sf,sfn,subflows,globalSubflows,activeNodes)
My subflow is::
[{"id":"4ff24a4d.e87f64","type":"subflow","name":"ipconfig","info":"","in":[{"x":162.50003814697266,"y":501.40003395080566,"wires":[{"id":"140d8d8a.0af372"}]}],"out":[{"x":612.1000137329102,"y":499.2000136375427,"wires":[{"id":"d049bacd.c36e98","port":0}]}]},{"id":"140d8d8a.0af372","type":"hostip","z":"4ff24a4d.e87f64","name":"Host IP","x":302.50003814697266,"y":501.40003395080566,"wires":[["d049bacd.c36e98"]]},{"id":"d049bacd.c36e98","type":"function","z":"4ff24a4d.e87f64","name":"Get IP","func":"\nvar ipv4 = msg.payload[0]\nmsg.payload = ipv4.address\nreturn msg;","outputs":1,"noerr":0,"x":482.10001373291016,"y":499.2000136375427,"wires":[[]]},{"id":"f4800a16.a32368","type":"subflow:4ff24a4d.e87f64","z":"55defe8b.eb37a","name":"","x":632,"y":234,"wires":[["1c73bf63.567651"]],"constraints":{"wolf":{"id":"wolf","deviceName":"wolf","fill":"#c66803","text":"wolf"}}}]
Here is the corresponding JSON of this flow:
[{"id":"23b7a311.c2d91c","type":"http in","z":"1b53be18.b0a122","name":"","url":"/test2","method":"get","upload":false,"swaggerDoc":"","x":120,"y":960,"wires":[["f62b16d5.355ec8"]],"constraints":{"connstraint1(device1-port12001)":{"id":"connstraint1(device1-port12001)","deviceName":"device1-port12001","fill":"#407266","text":"connstraint1(device1-port12001)"}}},{"id":"aa54ed4a.26107","type":"http response","z":"1b53be18.b0a122","name":"","statusCode":"","headers":{},"x":630,"y":960,"wires":[],"constraints":{"connstraint1(device1-port12001)":{"id":"connstraint1(device1-port12001)","deviceName":"device1-port12001","fill":"#407266","text":"connstraint1(device1-port12001)"}}},{"id":"f62b16d5.355ec8","type":"function","z":"1b53be18.b0a122","name":"LogOnly1","func":"// msg.payload = {\"a\":\"b\"};\nconsole.log(\"log1\");\nreturn msg;","outputs":1,"noerr":0,"x":340,"y":860,"wires":[["db91eed6.8e26"]],"constraints":{"connstraint2(device2-port12002)":{"id":"connstraint2(device2-port12002)","deviceName":"device2-port12002","fill":"#4286f4","text":"connstraint2(device2-port12002)"}}},{"id":"db91eed6.8e26","type":"function","z":"1b53be18.b0a122","name":"LogOnly2","func":"msg.payload = {\"a\":\"b\"};\nconsole.log(\"log2\");\nreturn msg;","outputs":1,"noerr":0,"x":440,"y":960,"wires":[["aa54ed4a.26107"]],"constraints":{}}]
When I visit http://localhost:12001/test2, the server (run on port 12001) does not reply a HTTP response.
There is a error 5 Aug 14:14:55 - [error] [dnr:ba71fbe6.1be358] TypeError: Converting circular structure to JSON
shown on Node-RED instance 1 (run on port 12001).
Node-RED version:0.18.7
node.js version: v8.11.2
npm version:6.1.0
Platform/OS: macOS 10.13.6 (17G65)
Browser: Google Chrome Version 67.0.3396.99 (Official Build) (64-bit)
The browser console complains of type error. Seems to have to do with Google api ...
127.0.0.1-1507661137633.txt
127.0.0.1-1507661232563.txt
CantReadProperty.js.txt
Apologies this is really just a question but I didn't see anywhere else to ask...
I think DNR is really exciting and I was just curious if this project is coordinating with the node-red project or is it a fork that is completely disconnected.
e.g. I was wondering are you committed to updating the DNR codebase as new releases of node-red come out? (I notice you are a little behind node-red right now but not a lot)
Is there any way that DNR would not have to be a fork of node-red in the future?
e.g. I notice node-red has "pluggable message routing" on their roadmap which I assume might have helped you implement DNR without having to fork node-red, but unfortunately I see they've removed "pluggable message routing" from the 1.0 roadmap.
(or looking the other way: have you guys made changes to node-red that might have become a PR for such pluggable routing?)
DNR is a really neat project thanks for doing it!!!
Darren
Problem 1:
The flow gets updated on the registered node-red devices but doesn't run reliably. It executes a couple of times with a and then stops. The Node-RED device asks for review and confirmation that flow should updated. This seems to result in the code not executing any further.
Problem 2:
Also the global flow received by registered device has no configs property resulting in access violation, which I hacked around. Here's an example of what's published by the coordinator to the Node-RED devices (publishTo in red\runtime\dnr.js)
"{"topic":"flow_deployed","data":{"activeFlow":{"id":"55defe8b.eb37a","label":"Flow 1","nodes":[{"id":"a8276f0b.23f18","type":"inject","z":"55defe8b.eb37a","name":"hh","topic":"gg","payload":"","payloadType":"date","repeat":"1","crontab":"","once":false,"x":506,"y":394,"wires":[["ce7e4dc0.fe619"]],"constraints":{"101":{"id":"101","deviceName":"fox","fill":"#f42404","text":"101"},"102":{"id":"102","deviceName":"wolf","fill":"#bf337b","text":"102"}}},{"id":"ce7e4dc0.fe619","type":"debug","z":"55defe8b.eb37a","name":"pq","active":true,"console":"false","complete":"payload","x":874,"y":351,"wires":[],"constraints":{"101":{"id":"101","deviceName":"fox","fill":"#f42404","text":"101"},"102":{"id":"102","deviceName":"wolf","fill":"#bf337b","text":"102"}}}]},"globalFlow":{"id":"global"},"allFlows":["55defe8b.eb37a"]}}"
For a new flow created in DNR editor that requires a new node to be installed, currently, that new module is only installed in the editor and not on each instance resulting in broken flows.
The proposed WIP Solution:
@namgk Had some good thoughts:
Deeper into the installation of node implementation, since the installation process might be complicated, such as npm build might occurs for native modules, this process is 1) lengthy, and 2) might fail. So we need to figure the strategy for this.
For example, if an installation is failed, should we retry it immediately? How about any back off timer till the next retry, maximum number of retries? Or simply just disable the whole flow on such device. We might need to implement some simple task queuing system for this before the full fledged flow can be up and running.
Further, if the missing node is device-agnostic, meaning it's computation does not depend on the device hosting it (e.g calculating maximum vs outputting current location), we also have an option to redirect the data to an external device (could be nearby) where the node is already installed and got the result back from there.
Is it possible to execute some flow in the dnr-editor instance?
For example, I would like to see the result in of the debug in the relative tab in the dnr-editor and not in a node-red instance of a remote (or local) device.
Thanks
Hi,
Thanks for the hard work on this, it shows a lot of promise.
I'm trying to set up a distributed workflow which emulates gnu parallel. In order to do this, I'd like to have a single command trigger processes on multiple nodes, each routing their results back to a single node.
Right now there seems to be no way to describe a single node as being executable by a set of devices simultaneously. Instead, I need to enumerate each device using a new node annotated with the deviceID as a constraint. I was under the impression that you could exclude the deviceID and it would send a message to all devices, but the message (say, a timestamp) doesn't arrive at every other node in spite of either 1N or NN link between them. I suspect this is because I am thinking in parallel and not as a matter of 1 message to 1 node, and that there is some intermediate node I should be using.
To reproduce, take three node-red instances, NR1, NR2 and NR3. Create a single timestamp node, an exec node (with ls as the command, excluding payload), and a debug node. Assign the timestamp node to NR1 (via node requirements) and set the exec node to a 1 1 requirement for mem/cores. Then set the debug node to N1.
Am I misunderstanding how to set up parallel flows?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.