flowbased / fbp-spec Goto Github PK

Data-driven testing of FBP components and graphs

License: MIT License

JavaScript 100.00%

test-runner testing tdd bdd fbp-protocol

fbp-spec's Introduction

fbp-spec

A runtime-independent test framework for Flow Based Programming (FBP) component and graphs, using declarative, data-driven testing.

One can use fbp-spec to do testing at multiple levels, each approximately corresponding to the different architectural levels of Flow Based Programming:

Unit (FBP component/subgraph)
Integration (FBP graph)
System (FBP runtime)

Status:

In production

Used by several projects, including Ingress Table and BIG IoT NoFlo bridge
Tested with several FBP runtimes: NoFlo, MicroFlo, Python example, MsgFlo
Runners available for contious integration (CLI, Mocha) and interactively (in Flowhub)

Purpose & Scope

Note: fbp-spec is intended for use by application and component-library developers.

The following is considered out-of-scope:

Testing conformance with the FBP protocol. Instead use fbp-protocol
Testing an FBP runtime/engine itself. Instead use a testing framework for your particular runtime language/environment.

License

The MIT license

Usage

Installing

Set up fbp-spec as an NPM dependency

npm install --save-dev fbp-spec

or, install it globally. Useful if you just want the commandline tool.

npm install -g fbp-spec

Writing tests

Each declared test suite loads an FBP component (or graph) fixture, and runs a set of test cases by sending a set of input data to input ports and verifying the output data against the expected results.

name: "Simple example of passing tests"
topic: "core/Repeat"
fixture:
 type: 'fbp'
 data: |
  INPORT=it.IN:IN
  OUTPORT=f.OUT:OUT
  it(core/Repeat) OUT -> IN f(core/Repeat)

cases:
-
  name: 'sending a boolean'
  assertion: 'should repeat the same'
  inputs:
    in: true
  expect:
    out:
      equals: true
-
  name: 'sending a number'
  assertion: 'should repeat the same'
  inputs:
    in: 1000
  expect:
    out:
      equals: 1000

Multiple ports

You can send data to multiple inports and check expectations on multiple ports per testcase:

-
  name: '1 active track toggled high'
  assertion: 'should give value1 color'
  inputs:
    tracks: 1
    animation: [
      0, # track idx
      "0xEE00EE", # val0
      "0xAA00AA", # val1
      200, # period
      50, # dutycycle
      0, # offset
      500 ] # duration
    clock: 250
  expect:
    clock:
     equals: 250
    value:
     equals: [0, 0x00AA] # FIXME: truncated

Sequence of packets

For testing components with state, you can sending multiple input packets in sequence.

-
  name: 'sequence of data using spacy notation'
  assertion: 'should pass'
  inputs:
    -
      in: true
    -
      in: false
  expect:
    - 
      out:
        equals: true
    -
      out:
        equals: false

Extract data using path

With path you can specify a JSONPath to extract the piece(s) of data the assertions will be ran against:

-
  name: 'select single value'
  assertion: 'should pass'
  inputs:
    in: { outer: { inner: { foo: 'bar' } } }
  expect:
    out:
      path: '$.outer.inner.foo'
      equals: 'bar'
-
  name: 'selecting many correct values'
  assertion: 'should pass'
  inputs:
    in:
      outer:
        first: { foo: 'bar' }
        second: { foo: 'bar' }
  expect:
    out:
      path: '$.outer.*.foo'
      equals: 'bar'

Skipping tests

Setting skip property on a testcase or suite, will cause it to not be ran. Should contain a message of the reason for skipping.

-
  name: 'a test that is skipped'
  assertion: 'will not be ran'
  inputs:
    in: 1000
  expect:
    out:
      equals: 1000
  skip: 'not implemented yet'

Using fixtures

One can use testing-specific components in the fixture, to simplify driving the unit under test with complex inputs and performing complex assertions.

fixture:
 type: 'fbp'
 data: |
  INPORT=imagename.IN:NAME
  INPORT=testee.PARAM:PARAM
  INPORT=reference.IN:REFERENCE
  OUTPORT=compare.OUT:SIMILARITY

  generate(test/GenerateTestImage) OUT -> IN testee(my/Component)
  testee OUT -> ACTUAL compare(test/CompareImage)
  reference(test/ReadReferenceImage) OUT -> REFERENCE compare
cases:
-
  name: 'testing complex data with custom components fixture'
  assertion: 'should pass'
  inputs:
    name: someimage
    param: 100
    reference: someimage-100-result
  expect:
    similarity:
      above: 0.99

Supported assertions

Instead of equals you can use any of the supported assertion predicates. Examples include:

type
above
below
contains
haveKeys
includeKeys

For a full set of assertions, see the schema

A comprehensive set of examples can be found under ./examples. For the detailed definition of the dataformat for tests, see schemata/.

Running tests with fbp-spec commandline tool

The simplest and most universal way of running tests is with the fbp-spec commandline tool.

$ fbp-spec --address ws://localhost:3333 examples/multisuite-failandpass.yaml

MultiSuite, failing tests
  sending a boolean with wrong expect
    should fail: ✗ Error: expected true to deeply equal false
  sending a number with wrong expect
    should fail: ✗ Error: expected 1000 to deeply equal 1003
MultiSuite, passing tests
  sending a boolean
    should repeat the same: ✓
  sending a number
    should repeat the same: ✓

The --command options can be used to specify a command which will start the runtime under test:

fbp-spec --command "python2 protocol-examples/python/runtime.py"

It sets the exit status to non-zero on failure, so is suitable for integrating into a Makefile or similar.

Running tests by integrating with Mocha

Mocha iss a popular test runner framework for JavaScript/CoffeeScript on browser and node.js.

Since fbp-spec communicates with your runtime over a network protocol, you can use this also when your project is not JavaScript-based. The Mocha runner is for instance used in microflo-core to test C++ components for microcontrollers & embedded devices.

You can have your fbp-spec tests run in Mocha by calling the fbpspec.mocha.run() function, in a file which is executed with the standard Mocha runner. Eg. mocha --reporter spec tests/fbpspecs.js

// fbpspecs.js
fbpspec = require('fbp-spec');

rt = {
  protocol: "websocket",
  address: "ws://localhost:3569",
  secret: "py3k", // Optional. If needed to connect/authenticate to runtime
  command: 'python2 protocol-examples/python/runtime.py' // Optional. Can be used to start runtime automatically
};
fbpspec.mocha.run(rt, './examples/simple-passing.yaml', { starttimeout: 1000 });

The tests can be specified as a list of files, or directories. You can use the standard grep option of Mocha to run only some tests.

For CoffeScript example, see ./spec/mocha.js.

Running tests interactively in Flowhub

Flowhub IDE (version 0.11 and later) has integrated support for fbp-spec. No installation is required.

Open existing project, or create a new one
Open a component, and write/copypaste in a test in the Tests panel
Ensure you have a runtime set up, and connected

When you make changes to your project (components,graphs) or tests, Flowhub will now automatically (re-)run your tests. You can see the status in the top-right corner. Clicking on it brings up more details.

Generating tests programatically

The test-format defined by fbp-spec is fairly generic and versatile. It is intended primarily as a format one directly specifies tests in, but can also be generated from other sources.

Sometimes data-driven testing, one does a large amount of very similar tests, with multiple test-cases per set of input data. By capturing only the unique parts of testcases in a specialied data-structure (JSON, YAML, etc), and then transforming this into standard fbp-spec files with some code, adding/removing cases becomes even easier. For instance in imgflo-server, testcases can be defined by providing a name, an URL and a reference result (a file with naming convention based on name).

Similarly, one can generate testcases using fuzzing, schema-based, model-based or similar tools.

Integrating test runner in an application

The test runner code is accessible as a JavaScript library, and can be integrated into other apps (like Flowhub does). See examples of commandline and webappp usage.

Add supporting for a new runtime

You need to implement the FBP network protocol. At least the protocol:runtime, protocol:graph, and protocol:network capabilities are required.

All transports supported by fbp-protocol-client are supported by fbp-spec, including WebSocket, WebRTC, and iframe/postMessage.

fbp-spec is intended to be used with flow-based and dataflow-programming, but might be useful also outside these programming paradigms. Try it out!

Writing a test runner in another language

As long as you stay compatible with the fbp-spec testformat and FBP protocol, you can implement a compatible runner in any programming language.

You can consider the fbp-spec code (in CoffeeScript) as a reference implementation.

fbp-spec's People

Contributors

Stargazers

Watchers

Forkers

derekk19 hkochitsonix cephdon standardgalactic

fbp-spec's Issues

UI: Make minimally useful

Show runtime connection status
Show tests running status. Ref #4
Show unexpected errors, both from runtime and from loading tests
Show assertion error of failing tests
Wire up runtime-provided tests in UI (or move to Runner). Currently only in CLI tool
Load directories of tests #68
Make the styling not be horrible

Maybe

Allow to edit the runtime connection info -> automatically updates URL
Support a grep/search option for running subset of tests

Support multiple sets of input and output messages

Needed for some (stateful) components. Like for instance in microflo-core HysteresisLatch:
https://github.com/microflo/microflo-core/blob/master/spec/HysteresisLatch.yaml#L48

Should let the inputs and expect keys optionally take an array of objects, in addition to the object directly.

Fetch flowtraces for testcase runs

With Flowtrace we have a way to persist traces of a network execution. fbp-spec should make use of this to allow debugging after failing test runs.

For each case, runner should set up tracing using trace:start. Then if test fails, trigger a trace:dump to retrieve it.
Should possibly also have an option to store trace even if test passes (per testcase). Trace from successful run can be useful to compare against a failure, or just to understand the program in general.

The CLI should maybe just dump these files to disk, with some meaningful names (derived from testcase names?). Can then use regular Flowtrace tools with them. Alternatively allow them to be uploaded somewhere (useful for CI).

An in-range update of fbp-protocol-client is breaking the build 🚨

Version 0.1.14 of fbp-protocol-client was just published.

Branch	Build failing 🚨
Dependency	fbp-protocol-client
Current Version	0.1.13
Type	dependency

This version is covered by your current version range and after updating it in your project the build failed.

fbp-protocol-client is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details

❌ continuous-integration/travis-ci/push The Travis CI build failed Details

Commits

The new version differs by 5 commits.

16bbfaa Bump
6de6222 Add safety
828a5bd Merge pull request #110 from flowbased/optional_debug
4306ece Use debug instead of console.log
d9fb026 Only output microflo availability message when debug is enabled

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.

Your Greenkeeper Bot 🌴

Allow to skip name/assertion, infer from input/expect

For instance if name is not specified, it could default to: "receving #{port}: #{data}"
If assertion is not specified, #{port} should #{predicate} #{data}. out should equal "foo"

This gets a little bit tricky with multiple ports, and sequences of this. But maybe an near plain JSON serialization would be fine. If people want it prettier, then should add the attributes themselves...

Setup should fail if trying to assert on a non-exported outport

In this case the fixture graph as an OUT outport, but the test tries to assert on HASH outport that doesn't exist.

The failure is:

Uncaught AssertionError: expected undefined to deeply equal 'QmYA...

Which is not super descriptive. Would be better to tell that such port doesn't exist.

Way to run tests against a browser runtime from Node.js

Right now a limitation of fbp-spec is that we don't really have a good way to test against a browser runtime from Node.js (like in Travis CI).

One idea would be spawning a postmessage runtime with chrome-headless and talking to it.

Generally this is out-of-scope for fbp-spec itself, but we need to implement a runtime example, so this issue is to track that.

Indicate test running status

fbpspec.runner.runAll() should enrich the testcases with started, stopped timestamps.
Probably also have a canonical state enumeration: pending, running, passed, failed, skipped, errored (for ).

Then expose this in the ui.widgets

Support for Chai .not

The chai .not expectation would be super useful, for example:

-
  name: 'listing hashes after removing'
  assertion: 'should not return the same hash'
  inputs:
    pinlist: true
  expect:
    out:
      not:
        contain: 'Qmctf39Vj8aLdg9icf2kvVcENyuNWBwy5TGWvHX9CZ6UkG'

Use case: the "listing hashes after removing" test in https://github.com/noflo/noflo-ipfs/blob/master/spec/Pin.yaml

If no fixture is specified, set up "topic" as the fixture graph

Will remove the need to specify fixture in the common cases where it is a single component/graph.

Error if runtime crashes

Right now it seems fbp-spec doesn't notice that it got disconnected from the runtime. This should cause the suite to fail instead of hang

Should fail with nice error when expecting against non-existing outport

fbp-spec-mocha: Support proxying to a runtime

Split out of #36.

To multiplex legacy cases and non-legacy cases across the same runtime connection.
It could be that the best approach is to let this mocha-legacy-compat tool proxy communication to the real runtime. Especially since this is a feature we want to have/build also for MsgFlo (msgflo/msgflo#28).

We also need to avoid double-detection of the proper fbp-spec cases when those are using the Mocha runner integration. Should be doable by that integraiton detecting when we are in this legacy mode.

Test coverage support

It would be nice to be able to get test coverage reports from fbp-spec. Something like https://www.npmjs.com/package/nyc

Tests times out if sending only on error

It is conventional to use a port named error to send errors, for instance in NoFlo components.
When causing errors, then the graph/component only sends on error - not the port for success (which fbp-spec is checking).
In this case we never receive any data (cause not listening for error) - and instead time out.
If we instead also automatically listened to error (if existing), we could check it and give a much more helpful error message.

Maybe support custom components inline

Right now all components must be existing in the runtime. For testing-specific components, it could maybe be nice to optionally allow these to be defined inline in the definition of the tests.

testsuite: Load directory of tests over HTTP

Need to parse typical index.html pages for (relative) links. If they have .yaml or .yml ending, try to load&parse as fbp-spec.
Non-recursive would be fine initially, people can specify test= multiple times if using multiple subdirectories.

testsuite: Allow loading from data:// url

Can be useful when using http://fbpspec.run

Error if process-error is sent

Only NoFlo implements this right now, but indicates an (unexpected) Exception. Normally these should be sent to error port (#71) but some components don't do this - and besides, bugs happen.

UI: Deploy as webapp

There is some UI code in ./ui, currently used by Flowhub (not released).
This could be a stateless "runner" app, especially now that we can fetch component tests from the runtime over the FBP protocol.

Respect URL parameter(s) for runtime connection info, similar to Flowhub live-mode
Take a URL parameter for fetching (additional) tests as http(s):// url
Automatically deployment to github pages
Get a nice domain

An in-range update of coffee-script is breaking the build 🚨

Version 1.12.6 of coffee-script just got published.

Branch	Build failing 🚨
Dependency	coffee-script
Current Version	1.12.5
Type	dependency

This version is covered by your current version range and after updating it in your project the build failed.

coffee-script is a direct dependency of this project this is very likely breaking your project right now. If other packages depend on you it’s very likely also breaking them.
I recommend you give this issue a very high priority. I’m sure you can resolve this 💪

Status Details

❌ continuous-integration/travis-ci/push The Travis CI build failed Details

Commits

The new version differs by 16 commits0.

f0e9837 1.12.6 missing updated output
f661f91 1.12.6 (#4548)
e00fa5d Fix #4533: chained calls incorrectly wrapping enclosing implicit objects (#4534)
51c0657 Fix #4150: Correctly outdent ternary followed by method call (#4535)
26cb24a return and export default can now accept implicit objects (#4532)
ac1b2b5 Iss4248 unicode code point escapes (cleanup) (#4522)
96b6c5f Fix #4248: Unicode code point escapes (#4498)
bfce054 Fix parenthesized conditions in if-else assignment (#4519)
ff60e6a fix 'future reserved words' test (#4518)
0da9d71 Fix broken links, update redirected links (#4505)
fecdbac A particular REPL test is broken in Node 4.8.2 because of a regression that was fixed in Node 5.11.0; just disable the test for Node < 6. Fixes #4502. (#4510)
473e8a1 Merge pull request #4507 from GeoffreyBooth/tests-exit-code
faf6d17 cake build:browser should just assemble the browser build, not also test it; add cake build:browser:full to additionally run the tests
d141d5c If the tests fail, return a non-zero exit code
a36b454 Merge branch 'master' of github.com:jashkenas/coffeescript

There are 16 commits in total.

See the full diff

Not sure how things should work exactly?

There is a collection of frequently asked questions and of course you may always ask my humans.

Your Greenkeeper Bot 🌴

Tests failing for browser

Also need to make the grunt-mocha-phantomjs task fail the build, so this gets caught in the future.

Handle FBP protocol error messages

Right now they are silently ignored.

An in-range update of bluebird is breaking the build 🚨

Version 3.5.1 of bluebird just got published.

Branch	Build failing 🚨
Dependency	bluebird
Current Version	3.5.0
Type	dependency

This version is covered by your current version range and after updating it in your project the build failed.

bluebird is a direct dependency of this project this is very likely breaking your project right now. If other packages depend on you it’s very likely also breaking them.
I recommend you give this issue a very high priority. I’m sure you can resolve this 💪

Status Details

❌ continuous-integration/travis-ci/push The Travis CI build failed Details

Release Notes

v3.5.1

Bugfixes:

Fix false positive unhandled rejection when using async await (#1404)
Fix false positive when reporting error as non-error (#990)

Commits

The new version differs by 26 commits.

dcfa52b Release v3.5.1
48c8591 Fixes #1404
3c93a91 Revert "Update default.html"
5155a7b Merge pull request #1463 from gabegorelick/isError
3f7ccf4 Better error check
f8f4a01 Merge pull request #1462 from themez/patch-1
18891c9 Fix title style
b2b14c2 Update default.html
a61aa1c tiny grammar change (#1457)
2c9f7a4 Update tap.md (#1438)
3a7fcbb fixes markdow headers (#1437)
e889c0d Correct a tiny tiny typo (#1431)
e03f12c improve .all() docs (#1436)
4f90934 Updated links (#1420)
3d2e57f Add Node.js 7+8 to Travis build matrix (#1412)

There are 26 commits in total.

See the full diff

Not sure how things should work exactly?

There is a collection of frequently asked questions and of course you may always ask my humans.

Your Greenkeeper Bot 🌴

Specifying files via glob only runs first file

If you run fbp-spec with:

$ fbp-spec --command "<cmd>" --address "<addr>" --secret foo spec/*.yaml

it only runs the first suite.

Workaround is to only specify dir:

$ fbp-spec --command "<cmd>" --address "<addr>" --secret foo spec

Large stacktrace when running fbp-spec following installation instructions

==== JS stack trace =========================================

Security context: 0x1d54076c9e59 <JS Object>#0#
    1: .node [module.js:568] [pc=0x3f37cae709e4] (this=0x3fd4c6006e99 <an Object with map 0x2ac0ae9d3109>#1#,module=0x2e63cb168859 <a Module with map 0x2ac0ae918319>#2#,filename=0x2e63cb1687e9 <String[87]: /Users/james/code/canadiannessgit/node_modules/serialport/build/Release/serialport.node>)
    2: load [/Users/james/code/canadiannessgit/node_modules/noflo/node_modules/coffee-script/lib/coffee-script/register.js:45] [pc=0x3f37cab41f34] (this=0x2e63cb168859 <a Module with map 0x2ac0ae918319>#2#,filename=0x2e63cb1687e9 <String[87]: /Users/james/code/canadiannessgit/node_modules/serialport/build/Release/serialport.node>)
    3: tryModuleLoad(aka tryModuleLoad) [module.js:417] [pc=0x3f37ca7342dd] (this=0x1d5407604189 <undefined>,module=0x2e63cb168859 <a Module with map 0x2ac0ae918319>#2#,filename=0x2e63cb1687e9 <String[87]: /Users/james/code/canadiannessgit/node_modules/serialport/build/Release/serialport.node>)
    4: _load [module.js:409] [pc=0x3f37ca72fec2] (this=0x3fd4c6006f19 <JS Function Module (SharedFunctionInfo 0x22319c824c31)>#3#,request=0x2e63cb167e49 <String[87]: /Users/james/code/canadiannessgit/node_modules/serialport/build/Release/serialport.node>,parent=0x2e63cb152749 <a Module with map 0x2ac0ae918319>#4#,isMain=0x1d5407604299 <false>)
    5: require [module.js:~465] [pc=0x3f37ca8191eb] (this=0x2e63cb152749 <a Module with map 0x2ac0ae918319>#4#,path=0x2e63cb167e49 <String[87]: /Users/james/code/canadiannessgit/node_modules/serialport/build/Release/serialport.node>)
    6: require(aka require) [internal/module.js:20] [pc=0x3f37ca73e206] (this=0x1d5407604189 <undefined>,path=0x2e63cb167e49 <String[87]: /Users/james/code/canadiannessgit/node_modules/serialport/build/Release/serialport.node>)
    7: bindings [/Users/james/code/canadiannessgit/node_modules/bindings/bindings.js:76] [pc=0x3f37caafb596] (this=0x1d54076e4ef9 <JS Global Object>#5#,opts=0x3c3185c6dc81 <String[15]: serialport.node>)
    8: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/serialport/lib/bindings.js:3] [pc=0x3f37caafa0ae] (this=0x2e63cb14cc99 <an Object with map 0xa3095b07bc9>#6#,exports=0x2e63cb14cc99 <an Object with map 0xa3095b07bc9>#6#,require=0x2e63cb14dc91 <JS Function require (SharedFunctionInfo 0x22319c8552a1)>#7#,module=0x2e63cb14cc49 <a Module with map 0x2ac0ae918319>#8#,__filename=0x2e63cb14cbe1 <String[73]: /Users/james/code/canadiannessgit/node_modules/serialport/lib/bindings.js>,__dirname=0x2e63cb14dc29 <String[61]: /Users/james/code/canadiannessgit/node_modules/serialport/lib>)
    9: _compile [module.js:541] [pc=0x3f37ca73d2b0] (this=0x2e63cb14cc49 <a Module with map 0x2ac0ae918319>#8#,content=0x2e63cb14d661 <String[784]\: 'use strict';\n\nvar bindings = require('bindings')('serialport.node');\nvar listUnix = require('./list-unix');\n\nvar linux = process.platform !== 'win32' && process.platform !== 'darwin';\n\nfunction listLinux(callback) {\n  callback = callback || function(err) {\n    if (err) { this.emit('error', err) }\n  }.bind(this);\n  return listUnix(callback);\n};\n\nvar platformOptions = {};\nif (process.platform !== 'win32') {\n  platformOptions = {\n    vmin: 1,\n    vtime: 0\n  };\n}\n\nmodule.exports = {\n  close: bindings.close,\n  drain: bindings.drain,\n  flush: bindings.flush,\n  list: linux ? listLinux : bindings.list,\n  open: bindings.open,\n  SerialportPoller: bindings.SerialportPoller,\n  set: bindings.set,\n  update: bindings.update,\n  write: bindings.write,\n  platformOptions: platformOptions\n};\n>,filename=0x2e63cb14cbe1 <String[73]: /Users/james/code/canadiannessgit/node_modules/serialport/lib/bindings.js>)
   10: .js [module.js:550] [pc=0x3f37ca735dab] (this=0x3fd4c6006e99 <an Object with map 0x2ac0ae9d3109>#1#,module=0x2e63cb14cc49 <a Module with map 0x2ac0ae918319>#8#,filename=0x2e63cb14cbe1 <String[73]: /Users/james/code/canadiannessgit/node_modules/serialport/lib/bindings.js>)
   11: load [/Users/james/code/canadiannessgit/node_modules/noflo/node_modules/coffee-script/lib/coffee-script/register.js:45] [pc=0x3f37cab41f34] (this=0x2e63cb14cc49 <a Module with map 0x2ac0ae918319>#8#,filename=0x2e63cb14cbe1 <String[73]: /Users/james/code/canadiannessgit/node_modules/serialport/lib/bindings.js>)
   12: tryModuleLoad(aka tryModuleLoad) [module.js:417] [pc=0x3f37ca7342dd] (this=0x1d5407604189 <undefined>,module=0x2e63cb14cc49 <a Module with map 0x2ac0ae918319>#8#,filename=0x2e63cb14cbe1 <String[73]: /Users/james/code/canadiannessgit/node_modules/serialport/lib/bindings.js>)
   13: _load [module.js:~383] [pc=0x3f37cad79cd7] (this=0x3fd4c6006f19 <JS Function Module (SharedFunctionInfo 0x22319c824c31)>#3#,request=0x3c3185c54741 <String[10]: ./bindings>,parent=0x2e63cb0ff629 <a Module with map 0x2ac0ae918319>#9#,isMain=0x1d5407604299 <false>)
   14: require [module.js:~465] [pc=0x3f37ca8191eb] (this=0x2e63cb0ff629 <a Module with map 0x2ac0ae918319>#9#,path=0x3c3185c54741 <String[10]: ./bindings>)
   15: require(aka require) [internal/module.js:20] [pc=0x3f37ca73e206] (this=0x1d5407604189 <undefined>,path=0x3c3185c54741 <String[10]: ./bindings>)
   16: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/serialport/lib/serialport.js:16] [pc=0x3f37caefdaae] (this=0x2e63cb0ff679 <an Object with map 0xa3095b07bc9>#10#,exports=0x2e63cb0ff679 <an Object with map 0xa3095b07bc9>#10#,require=0x2e63cb1081f9 <JS Function require (SharedFunctionInfo 0x22319c8552a1)>#11#,module=0x2e63cb0ff629 <a Module with map 0x2ac0ae918319>#9#,__filename=0x2e63cb0ff5c1 <String[75]: /Users/james/code/canadiannessgit/node_modules/serialport/lib/serialport.js>,__dirname=0x2e63cb108191 <String[61]: /Users/james/code/canadiannessgit/node_modules/serialport/lib>)
   17: _compile [module.js:541] [pc=0x3f37ca73d2b0] (this=0x2e63cb0ff629 <a Module with map 0x2ac0ae918319>#9#,content=0x2e63cb104271 <Very long string[13557]>#12#,filename=0x2e63cb0ff5c1 <String[75]: /Users/james/code/canadiannessgit/node_modules/serialport/lib/serialport.js>)
   18: .js [module.js:550] [pc=0x3f37ca735dab] (this=0x3fd4c6006e99 <an Object with map 0x2ac0ae9d3109>#1#,module=0x2e63cb0ff629 <a Module with map 0x2ac0ae918319>#9#,filename=0x2e63cb0ff5c1 <String[75]: /Users/james/code/canadiannessgit/node_modules/serialport/lib/serialport.js>)
   19: load [/Users/james/code/canadiannessgit/node_modules/noflo/node_modules/coffee-script/lib/coffee-script/register.js:45] [pc=0x3f37cab41f34] (this=0x2e63cb0ff629 <a Module with map 0x2ac0ae918319>#9#,filename=0x2e63cb0ff5c1 <String[75]: /Users/james/code/canadiannessgit/node_modules/serialport/lib/serialport.js>)
   20: tryModuleLoad(aka tryModuleLoad) [module.js:417] [pc=0x3f37ca7342dd] (this=0x1d5407604189 <undefined>,module=0x2e63cb0ff629 <a Module with map 0x2ac0ae918319>#9#,filename=0x2e63cb0ff5c1 <String[75]: /Users/james/code/canadiannessgit/node_modules/serialport/lib/serialport.js>)
   21: _load [module.js:~383] [pc=0x3f37cad79cd7] (this=0x3fd4c6006f19 <JS Function Module (SharedFunctionInfo 0x22319c824c31)>#3#,request=0x9323a0396a1 <String[10]: serialport>,parent=0x2e63cb006059 <a Module with map 0x2ac0ae918319>#13#,isMain=0x1d5407604299 <false>)
   22: require [module.js:~465] [pc=0x3f37ca8191eb] (this=0x2e63cb006059 <a Module with map 0x2ac0ae918319>#13#,path=0x9323a0396a1 <String[10]: serialport>)
   23: require(aka require) [internal/module.js:20] [pc=0x3f37ca73e206] (this=0x1d5407604189 <undefined>,path=0x9323a0396a1 <String[10]: serialport>)
   24: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/microflo/lib/serial.coffee:18] [pc=0x3f37caefd2d8] (this=0x2e63cb0167f1 <an Object with map 0xa3095b07bc9>#14#)
   25: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/microflo/lib/serial.coffee:189] [pc=0x3f37caefd70d] (this=0x2e63cb0167f1 <an Object with map 0xa3095b07bc9>#14#,exports=0x2e63cb0167f1 <an Object with map 0xa3095b07bc9>#14#,require=0x2e63cb0f77f1 <JS Function require (SharedFunctionInfo 0x22319c8552a1)>#15#,module=0x2e63cb006059 <a Module with map 0x2ac0ae918319>#13#,__filename=0x2e63cb005ff1 <String[73]: /Users/james/code/canadiannessgit/node_modules/microflo/lib/serial.coffee>,__dirname=0x2e63cb0f7789 <String[59]: /Users/james/code/canadiannessgit/node_modules/microflo/lib>)
   26: _compile [module.js:541] [pc=0x3f37ca73d2b0] (this=0x2e63cb006059 <a Module with map 0x2ac0ae918319>#13#,content=0x2e63cb0f4649 <Very long string[5611]>#16#,filename=0x2e63cb005ff1 <String[73]: /Users/james/code/canadiannessgit/node_modules/microflo/lib/serial.coffee>)
   27: .coffee [/Users/james/code/canadiannessgit/node_modules/noflo/node_modules/coffee-script/lib/coffee-script/register.js:16] [pc=0x3f37cab424b4] (this=0x3fd4c6006e99 <an Object with map 0x2ac0ae9d3109>#1#,module=0x2e63cb006059 <a Module with map 0x2ac0ae918319>#13#,filename=0x2e63cb005ff1 <String[73]: /Users/james/code/canadiannessgit/node_modules/microflo/lib/serial.coffee>)
   28: load [/Users/james/code/canadiannessgit/node_modules/noflo/node_modules/coffee-script/lib/coffee-script/register.js:45] [pc=0x3f37cab41f34] (this=0x2e63cb006059 <a Module with map 0x2ac0ae918319>#13#,filename=0x2e63cb005ff1 <String[73]: /Users/james/code/canadiannessgit/node_modules/microflo/lib/serial.coffee>)
   29: tryModuleLoad(aka tryModuleLoad) [module.js:417] [pc=0x3f37ca7342dd] (this=0x1d5407604189 <undefined>,module=0x2e63cb006059 <a Module with map 0x2ac0ae918319>#13#,filename=0x2e63cb005ff1 <String[73]: /Users/james/code/canadiannessgit/node_modules/microflo/lib/serial.coffee>)
   30: _load [module.js:~383] [pc=0x3f37cad79cd7] (this=0x3fd4c6006f19 <JS Function Module (SharedFunctionInfo 0x22319c824c31)>#3#,request=0x9323a03cdf9 <String[8]: ./serial>,parent=0x2699a6e32a81 <a Module with map 0x2ac0ae918319>#17#,isMain=0x1d5407604299 <false>)
   31: require [module.js:~465] [pc=0x3f37ca8191eb] (this=0x2699a6e32a81 <a Module with map 0x2ac0ae918319>#17#,path=0x9323a03cdf9 <String[8]: ./serial>)
   32: require(aka require) [internal/module.js:20] [pc=0x3f37ca73e206] (this=0x1d5407604189 <undefined>,path=0x9323a03cdf9 <String[8]: ./serial>)
   33: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/microflo/lib/runtime.coffee:34] [pc=0x3f37cae4916e] (this=0x2699a6e696d9 <an Object with map 0xa3095b07bc9>#18#)
   34: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/microflo/lib/runtime.coffee:798] [pc=0x3f37cae49c6d] (this=0x2699a6e696d9 <an Object with map 0xa3095b07bc9>#18#,exports=0x2699a6e696d9 <an Object with map 0xa3095b07bc9>#18#,require=0x308f1165f161 <JS Function require (SharedFunctionInfo 0x22319c8552a1)>#19#,module=0x2699a6e32a81 <a Module with map 0x2ac0ae918319>#17#,__filename=0x2699a6e32a19 <String[74]: /Users/james/code/canadiannessgit/node_modules/microflo/lib/runtime.coffee>,__dirname=0x308f1165f3b9 <String[59]: /Users/james/code/canadiannessgit/node_modules/microflo/lib>)
   35: _compile [module.js:541] [pc=0x3f37ca73d2b0] (this=0x2699a6e32a81 <a Module with map 0x2ac0ae918319>#17#,content=0x308f1166b0c1 <Very long string[24079]>#20#,filename=0x2699a6e32a19 <String[74]: /Users/james/code/canadiannessgit/node_modules/microflo/lib/runtime.coffee>)
   36: .coffee [/Users/james/code/canadiannessgit/node_modules/noflo/node_modules/coffee-script/lib/coffee-script/register.js:16] [pc=0x3f37cab424b4] (this=0x3fd4c6006e99 <an Object with map 0x2ac0ae9d3109>#1#,module=0x2699a6e32a81 <a Module with map 0x2ac0ae918319>#17#,filename=0x2699a6e32a19 <String[74]: /Users/james/code/canadiannessgit/node_modules/microflo/lib/runtime.coffee>)
   37: load [/Users/james/code/canadiannessgit/node_modules/noflo/node_modules/coffee-script/lib/coffee-script/register.js:45] [pc=0x3f37cab41f34] (this=0x2699a6e32a81 <a Module with map 0x2ac0ae918319>#17#,filename=0x2699a6e32a19 <String[74]: /Users/james/code/canadiannessgit/node_modules/microflo/lib/runtime.coffee>)
   38: tryModuleLoad(aka tryModuleLoad) [module.js:417] [pc=0x3f37ca7342dd] (this=0x1d5407604189 <undefined>,module=0x2699a6e32a81 <a Module with map 0x2ac0ae918319>#17#,filename=0x2699a6e32a19 <String[74]: /Users/james/code/canadiannessgit/node_modules/microflo/lib/runtime.coffee>)
   39: _load [module.js:~383] [pc=0x3f37cad79cd7] (this=0x3fd4c6006f19 <JS Function Module (SharedFunctionInfo 0x22319c824c31)>#3#,request=0x9323a03ccd1 <String[9]: ./runtime>,parent=0x1b62b71bf369 <a Module with map 0x2ac0ae918319>#21#,isMain=0x1d5407604299 <false>)
   40: require [module.js:~465] [pc=0x3f37ca8191eb] (this=0x1b62b71bf369 <a Module with map 0x2ac0ae918319>#21#,path=0x9323a03ccd1 <String[9]: ./runtime>)
   41: require(aka require) [internal/module.js:20] [pc=0x3f37ca73e206] (this=0x1d5407604189 <undefined>,path=0x9323a03ccd1 <String[9]: ./runtime>)
   42: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/microflo/lib/microflo.coffee:5] [pc=0x3f37cade83b4] (this=0x1b62b71bf521 <an Object with map 0xa3095b07bc9>#22#)
   43: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/microflo/lib/microflo.coffee:13] [pc=0x3f37cade86ad] (this=0x1b62b71bf521 <an Object with map 0xa3095b07bc9>#22#,exports=0x1b62b71bf521 <an Object with map 0xa3095b07bc9>#22#,require=0x1b62b71bf3f9 <JS Function require (SharedFunctionInfo 0x22319c8552a1)>#23#,module=0x1b62b71bf369 <a Module with map 0x2ac0ae918319>#21#,__filename=0x1b62b71bf5c9 <String[75]: /Users/james/code/canadiannessgit/node_modules/microflo/lib/microflo.coffee>,__dirname=0x1b62b71bf5a1 <String[59]: /Users/james/code/canadiannessgit/node_modules/microflo/lib>)
   44: _compile [module.js:541] [pc=0x3f37ca73d2b0] (this=0x1b62b71bf369 <a Module with map 0x2ac0ae918319>#21#,content=0x1b62b71bf829 <String[379]\: (function() {\n  module.exports = {\n    util: require("./util"),\n    componentlib: require("./componentlib"),\n    runtime: require("./runtime"),\n    generate: require("./generate"),\n    commandstream: require("./commandstream"),\n    devicecommunication: require("./devicecommunication"),\n    simulator: require("./simulator"),\n    serial: require("./serial")\n  };\n\n}).call(this);\n>,filename=0x1b62b71bf5c9 <String[75]: /Users/james/code/canadiannessgit/node_modules/microflo/lib/microflo.coffee>)
   45: .coffee [/Users/james/code/canadiannessgit/node_modules/noflo/node_modules/coffee-script/lib/coffee-script/register.js:16] [pc=0x3f37cab424b4] (this=0x3fd4c6006e99 <an Object with map 0x2ac0ae9d3109>#1#,module=0x1b62b71bf369 <a Module with map 0x2ac0ae918319>#21#,filename=0x1b62b71bf5c9 <String[75]: /Users/james/code/canadiannessgit/node_modules/microflo/lib/microflo.coffee>)
   46: load [/Users/james/code/canadiannessgit/node_modules/noflo/node_modules/coffee-script/lib/coffee-script/register.js:45] [pc=0x3f37cab41f34] (this=0x1b62b71bf369 <a Module with map 0x2ac0ae918319>#21#,filename=0x1b62b71bf5c9 <String[75]: /Users/james/code/canadiannessgit/node_modules/microflo/lib/microflo.coffee>)
   47: tryModuleLoad(aka tryModuleLoad) [module.js:417] [pc=0x3f37ca7342dd] (this=0x1d5407604189 <undefined>,module=0x1b62b71bf369 <a Module with map 0x2ac0ae918319>#21#,filename=0x1b62b71bf5c9 <String[75]: /Users/james/code/canadiannessgit/node_modules/microflo/lib/microflo.coffee>)
   48: _load [module.js:~383] [pc=0x3f37cad79cd7] (this=0x3fd4c6006f19 <JS Function Module (SharedFunctionInfo 0x22319c824c31)>#3#,request=0x109346653891 <String[8]: microflo>,parent=0x9323a0a8a79 <a Module with map 0x2ac0ae918319>#24#,isMain=0x1d5407604299 <false>)
   49: require [module.js:~465] [pc=0x3f37ca8191eb] (this=0x9323a0a8a79 <a Module with map 0x2ac0ae918319>#24#,path=0x109346653891 <String[8]: microflo>)
   50: require(aka require) [internal/module.js:20] [pc=0x3f37ca73e206] (this=0x1d5407604189 <undefined>,path=0x109346653891 <String[8]: microflo>)
   51: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/fbp-protocol-client/src/microflo.coffee:9] [pc=0x3f37cade782c] (this=0x9323a0a8ac9 <an Object with map 0xa3095b07bc9>#25#)
   52: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/fbp-protocol-client/src/microflo.coffee:262] [pc=0x3f37cade7acd] (this=0x9323a0a8ac9 <an Object with map 0xa3095b07bc9>#25#,exports=0x9323a0a8ac9 <an Object with map 0xa3095b07bc9>#25#,require=0x1b62b71bfa21 <JS Function require (SharedFunctionInfo 0x22319c8552a1)>#26#,module=0x9323a0a8a79 <a Module with map 0x2ac0ae918319>#24#,__filename=0x9323a0a8b01 <String[86]: /Users/james/code/canadiannessgit/node_modules/fbp-protocol-client/src/microflo.coffee>,__dirname=0x1b62b71bfb99 <String[70]: /Users/james/code/canadiannessgit/node_modules/fbp-protocol-client/src>)
   53: _compile [module.js:541] [pc=0x3f37ca73d2b0] (this=0x9323a0a8a79 <a Module with map 0x2ac0ae918319>#24#,content=0x1b62b71c3ae9 <Very long string[7990]>#27#,filename=0x9323a0a8b01 <String[86]: /Users/james/code/canadiannessgit/node_modules/fbp-protocol-client/src/microflo.coffee>)
   54: .coffee [/Users/james/code/canadiannessgit/node_modules/noflo/node_modules/coffee-script/lib/coffee-script/register.js:16] [pc=0x3f37cab424b4] (this=0x3fd4c6006e99 <an Object with map 0x2ac0ae9d3109>#1#,module=0x9323a0a8a79 <a Module with map 0x2ac0ae918319>#24#,filename=0x9323a0a8b01 <String[86]: /Users/james/code/canadiannessgit/node_modules/fbp-protocol-client/src/microflo.coffee>)
   55: load [/Users/james/code/canadiannessgit/node_modules/noflo/node_modules/coffee-script/lib/coffee-script/register.js:45] [pc=0x3f37cab41f34] (this=0x9323a0a8a79 <a Module with map 0x2ac0ae918319>#24#,filename=0x9323a0a8b01 <String[86]: /Users/james/code/canadiannessgit/node_modules/fbp-protocol-client/src/microflo.coffee>)
   56: tryModuleLoad(aka tryModuleLoad) [module.js:417] [pc=0x3f37ca7342dd] (this=0x1d5407604189 <undefined>,module=0x9323a0a8a79 <a Module with map 0x2ac0ae918319>#24#,filename=0x9323a0a8b01 <String[86]: /Users/james/code/canadiannessgit/node_modules/fbp-protocol-client/src/microflo.coffee>)
   57: _load [module.js:~383] [pc=0x3f37cad79cd7] (this=0x3fd4c6006f19 <JS Function Module (SharedFunctionInfo 0x22319c824c31)>#3#,request=0x109346653f09 <String[14]: ./src/microflo>,parent=0x3275446303d1 <a Module with map 0x2ac0ae918319>#28#,isMain=0x1d5407604299 <false>)
   58: require [module.js:~465] [pc=0x3f37ca8191eb] (this=0x3275446303d1 <a Module with map 0x2ac0ae918319>#28#,path=0x109346653f09 <String[14]: ./src/microflo>)
   59: require(aka require) [internal/module.js:20] [pc=0x3f37ca73e206] (this=0x1d5407604189 <undefined>,path=0x109346653f09 <String[14]: ./src/microflo>)
   60: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/fbp-protocol-client/index.js:8] [pc=0x3f37cad7a96a] (this=0x3275446304d9 <an Object with map 0x2ac0ae9885d1>#29#,exports=0x3275446304d9 <an Object with map 0x2ac0ae9885d1>#29#,require=0x327544630461 <JS Function require (SharedFunctionInfo 0x22319c8552a1)>#30#,module=0x3275446303d1 <a Module with map 0x2ac0ae918319>#28#,__filename=0x3275446305b9 <String[75]: /Users/james/code/canadiannessgit/node_modules/fbp-protocol-client/index.js>,__dirname=0x327544630591 <String[66]: /Users/james/code/canadiannessgit/node_modules/fbp-protocol-client>)
   61: _compile [module.js:541] [pc=0x3f37ca73d2b0] (this=0x3275446303d1 <a Module with map 0x2ac0ae918319>#28#,content=0x327544630829 <String[398]\: exports.transports = {\n  'websocket': require('./src/websocket'),\n  'iframe': require('./src/iframe'),\n  'webrtc': require('./src/webrtc')\n};\n\ntry {\n  exports.transports.microflo = require('./src/microflo');\n} catch (e) {\n  console.log('fbp-protocol-client: MicroFlo transport unavailable: ' + e.message);\n}\n\nexports.getTransport = function (transport) {\n  return exports.transports[transport];\n};\n>,filename=0x3275446305b9 <String[75]: /Users/james/code/canadiannessgit/node_modules/fbp-protocol-client/index.js>)
   62: .js [module.js:550] [pc=0x3f37ca735dab] (this=0x3fd4c6006e99 <an Object with map 0x2ac0ae9d3109>#1#,module=0x3275446303d1 <a Module with map 0x2ac0ae918319>#28#,filename=0x3275446305b9 <String[75]: /Users/james/code/canadiannessgit/node_modules/fbp-protocol-client/index.js>)
   63: load [/Users/james/code/canadiannessgit/node_modules/noflo/node_modules/coffee-script/lib/coffee-script/register.js:45] [pc=0x3f37cab41f34] (this=0x3275446303d1 <a Module with map 0x2ac0ae918319>#28#,filename=0x3275446305b9 <String[75]: /Users/james/code/canadiannessgit/node_modules/fbp-protocol-client/index.js>)
   64: tryModuleLoad(aka tryModuleLoad) [module.js:417] [pc=0x3f37ca7342dd] (this=0x1d5407604189 <undefined>,module=0x3275446303d1 <a Module with map 0x2ac0ae918319>#28#,filename=0x3275446305b9 <String[75]: /Users/james/code/canadiannessgit/node_modules/fbp-protocol-client/index.js>)
   65: _load [module.js:~383] [pc=0x3f37cad79cd7] (this=0x3fd4c6006f19 <JS Function Module (SharedFunctionInfo 0x22319c824c31)>#3#,request=0x2254a43809c9 <String[19]: fbp-protocol-client>,parent=0x25984467db41 <a Module with map 0x2ac0ae918319>#31#,isMain=0x1d5407604299 <false>)
   66: require [module.js:~465] [pc=0x3f37ca8191eb] (this=0x25984467db41 <a Module with map 0x2ac0ae918319>#31#,path=0x2254a43809c9 <String[19]: fbp-protocol-client>)
   67: require(aka require) [internal/module.js:20] [pc=0x3f37ca73e206] (this=0x1d5407604189 <undefined>,path=0x2254a43809c9 <String[19]: fbp-protocol-client>)
   68: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/fbp-spec/src/runner.coffee:15] [pc=0x3f37cad1a62c] (this=0x1cc8d2317231 <an Object with map 0xa3095b07bc9>#32#)
   69: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/fbp-spec/src/runner.coffee:465] [pc=0x3f37cad1abed] (this=0x1cc8d2317231 <an Object with map 0xa3095b07bc9>#32#,exports=0x1cc8d2317231 <an Object with map 0xa3095b07bc9>#32#,require=0x383b6ddf12e1 <JS Function require (SharedFunctionInfo 0x22319c8552a1)>#33#,module=0x25984467db41 <a Module with map 0x2ac0ae918319>#31#,__filename=0x25984467dad9 <String[73]: /Users/james/code/canadiannessgit/node_modules/fbp-spec/src/runner.coffee>,__dirname=0x383b6ddf14a9 <String[59]: /Users/james/code/canadiannessgit/node_modules/fbp-spec/src>)
   70: _compile [module.js:541] [pc=0x3f37ca73d2b0] (this=0x25984467db41 <a Module with map 0x2ac0ae918319>#31#,content=0x383b6ddf85b9 <Very long string[14356]>#34#,filename=0x25984467dad9 <String[73]: /Users/james/code/canadiannessgit/node_modules/fbp-spec/src/runner.coffee>)
   71: .coffee [/Users/james/code/canadiannessgit/node_modules/noflo/node_modules/coffee-script/lib/coffee-script/register.js:16] [pc=0x3f37cab424b4] (this=0x3fd4c6006e99 <an Object with map 0x2ac0ae9d3109>#1#,module=0x25984467db41 <a Module with map 0x2ac0ae918319>#31#,filename=0x25984467dad9 <String[73]: /Users/james/code/canadiannessgit/node_modules/fbp-spec/src/runner.coffee>)
   72: load [/Users/james/code/canadiannessgit/node_modules/noflo/node_modules/coffee-script/lib/coffee-script/register.js:45] [pc=0x3f37cab41f34] (this=0x25984467db41 <a Module with map 0x2ac0ae918319>#31#,filename=0x25984467dad9 <String[73]: /Users/james/code/canadiannessgit/node_modules/fbp-spec/src/runner.coffee>)
   73: tryModuleLoad(aka tryModuleLoad) [module.js:417] [pc=0x3f37ca7342dd] (this=0x1d5407604189 <undefined>,module=0x25984467db41 <a Module with map 0x2ac0ae918319>#31#,filename=0x25984467dad9 <String[73]: /Users/james/code/canadiannessgit/node_modules/fbp-spec/src/runner.coffee>)
   74: _load [module.js:409] [pc=0x3f37ca72fec2] (this=0x3fd4c6006f19 <JS Function Module (SharedFunctionInfo 0x22319c824c31)>#3#,request=0x249502bc9b91 <String[8]: ./runner>,parent=0x25984467dbb1 <a Module with map 0x2ac0ae918319>#35#,isMain=0x1d5407604299 <false>)
   75: require [module.js:~465] [pc=0x3f37ca8191eb] (this=0x25984467dbb1 <a Module with map 0x2ac0ae918319>#35#,path=0x249502bc9b91 <String[8]: ./runner>)
   76: require(aka require) [internal/module.js:20] [pc=0x3f37ca73e206] (this=0x1d5407604189 <undefined>,path=0x249502bc9b91 <String[8]: ./runner>)
   77: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/fbp-spec/src/index.coffee:12] [pc=0x3f37caccb997] (this=0x25984467dda9 <an Object with map 0xa3095b07bc9>#36#)
   78: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/fbp-spec/src/index.coffee:23] [pc=0x3f37caccbd0d] (this=0x25984467dda9 <an Object with map 0xa3095b07bc9>#36#,exports=0x25984467dda9 <an Object with map 0xa3095b07bc9>#36#,require=0x25984467dc41 <JS Function require (SharedFunctionInfo 0x22319c8552a1)>#37#,module=0x25984467dbb1 <a Module with map 0x2ac0ae918319>#35#,__filename=0x25984467de51 <String[72]: /Users/james/code/canadiannessgit/node_modules/fbp-spec/src/index.coffee>,__dirname=0x25984467de29 <String[59]: /Users/james/code/canadiannessgit/node_modules/fbp-spec/src>)
   79: _compile [module.js:541] [pc=0x3f37ca73d2b0] (this=0x25984467dbb1 <a Module with map 0x2ac0ae918319>#35#,content=0x25984467e109 <String[478]\: (function() {\n  var getSchema;\n\n  getSchema = function(id) {\n    var schema;\n    id = id.replace('.json', '');\n    schema = module.exports.schema[id];\n    return schema;\n  };\n\n  module.exports = {\n    runner: require('./runner'),\n    subprocess: require('./subprocess'),\n    mocha: require('./mocha'),\n    testsuite: require('./testsuite'),\n    schema: require('../schema'),\n    getSchema: getSchema,\n    ui: {\n      widgets: require('../ui/widgets')\n    }\n  };\n\n}).call(this);\n>,filename=0x25984467de51 <String[72]: /Users/james/code/canadiannessgit/node_modules/fbp-spec/src/index.coffee>)
   80: .coffee [/Users/james/code/canadiannessgit/node_modules/noflo/node_modules/coffee-script/lib/coffee-script/register.js:16] [pc=0x3f37cab424b4] (this=0x3fd4c6006e99 <an Object with map 0x2ac0ae9d3109>#1#,module=0x25984467dbb1 <a Module with map 0x2ac0ae918319>#35#,filename=0x25984467de51 <String[72]: /Users/james/code/canadiannessgit/node_modules/fbp-spec/src/index.coffee>)
   81: load [/Users/james/code/canadiannessgit/node_modules/noflo/node_modules/coffee-script/lib/coffee-script/register.js:45] [pc=0x3f37cab41f34] (this=0x25984467dbb1 <a Module with map 0x2ac0ae918319>#35#,filename=0x25984467de51 <String[72]: /Users/james/code/canadiannessgit/node_modules/fbp-spec/src/index.coffee>)
   82: tryModuleLoad(aka tryModuleLoad) [module.js:417] [pc=0x3f37ca7342dd] (this=0x1d5407604189 <undefined>,module=0x25984467dbb1 <a Module with map 0x2ac0ae918319>#35#,filename=0x25984467de51 <String[72]: /Users/james/code/canadiannessgit/node_modules/fbp-spec/src/index.coffee>)
   83: _load [module.js:409] [pc=0x3f37ca72fec2] (this=0x3fd4c6006f19 <JS Function Module (SharedFunctionInfo 0x22319c824c31)>#3#,request=0x2254a4380f31 <String[11]: ./src/index>,parent=0x25984467e321 <a Module with map 0x2ac0ae918319>#38#,isMain=0x1d5407604299 <false>)
   84: require [module.js:~465] [pc=0x3f37ca8191eb] (this=0x25984467e321 <a Module with map 0x2ac0ae918319>#38#,path=0x2254a4380f31 <String[11]: ./src/index>)
   85: require(aka require) [internal/module.js:20] [pc=0x3f37ca73e206] (this=0x1d5407604189 <undefined>,path=0x2254a4380f31 <String[11]: ./src/index>)
   86: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/fbp-spec/index.js:2] [pc=0x3f37caccb56d] (this=0x25984467e4c1 <an Object with map 0xa3095b07bc9>#39#,exports=0x25984467e4c1 <an Object with map 0xa3095b07bc9>#39#,require=0x25984467e3b1 <JS Function require (SharedFunctionInfo 0x22319c8552a1)>#40#,module=0x25984467e321 <a Module with map 0x2ac0ae918319>#38#,__filename=0x25984467e469 <String[64]: /Users/james/code/canadiannessgit/node_modules/fbp-spec/index.js>,__dirname=0x25984467e441 <String[55]: /Users/james/code/canadiannessgit/node_modules/fbp-spec>)
   87: _compile [module.js:541] [pc=0x3f37ca73d2b0] (this=0x25984467e321 <a Module with map 0x2ac0ae918319>#38#,content=0x25984467e5c1 <String[76]\: require('coffee-script/register');\nmodule.exports = require('./src/index');\n>,filename=0x25984467e469 <String[64]: /Users/james/code/canadiannessgit/node_modules/fbp-spec/index.js>)
   88: .js [module.js:550] [pc=0x3f37ca735dab] (this=0x3fd4c6006e99 <an Object with map 0x2ac0ae9d3109>#1#,module=0x25984467e321 <a Module with map 0x2ac0ae918319>#38#,filename=0x25984467e469 <String[64]: /Users/james/code/canadiannessgit/node_modules/fbp-spec/index.js>)
   89: load [/Users/james/code/canadiannessgit/node_modules/noflo/node_modules/coffee-script/lib/coffee-script/register.js:45] [pc=0x3f37cab41f34] (this=0x25984467e321 <a Module with map 0x2ac0ae918319>#38#,filename=0x25984467e469 <String[64]: /Users/james/code/canadiannessgit/node_modules/fbp-spec/index.js>)
   90: tryModuleLoad(aka tryModuleLoad) [module.js:417] [pc=0x3f37ca7342dd] (this=0x1d5407604189 <undefined>,module=0x25984467e321 <a Module with map 0x2ac0ae918319>#38#,filename=0x25984467e469 <String[64]: /Users/james/code/canadiannessgit/node_modules/fbp-spec/index.js>)
   91: _load [module.js:409] [pc=0x3f37ca72fec2] (this=0x3fd4c6006f19 <JS Function Module (SharedFunctionInfo 0x22319c824c31)>#3#,request=0x249502bb31b9 <String[8]: fbp-spec>,parent=0x25984467e649 <a Module with map 0x2ac0ae918319>#41#,isMain=0x1d5407604299 <false>)
   92: require [module.js:~465] [pc=0x3f37ca8191eb] (this=0x25984467e649 <a Module with map 0x2ac0ae918319>#41#,path=0x249502bb31b9 <String[8]: fbp-spec>)
   93: require(aka require) [internal/module.js:20] [pc=0x3f37ca73e206] (this=0x1d5407604189 <undefined>,path=0x249502bb31b9 <String[8]: fbp-spec>)
   94: /* anonymous */ [/Users/james/code/canadiannessgit/spec/fbpspec.coffee:4] [pc=0x3f37caccb124] (this=0x25984467e7a1 <an Object with map 0xa3095b07bc9>#42#)
   95: /* anonymous */ [/Users/james/code/canadiannessgit/spec/fbpspec.coffee:22] [pc=0x3f37caccb345] (this=0x25984467e7a1 <an Object with map 0xa3095b07bc9>#42#,exports=0x25984467e7a1 <an Object with map 0xa3095b07bc9>#42#,require=0x25984467e6d9 <JS Function require (SharedFunctionInfo 0x22319c8552a1)>#43#,module=0x25984467e649 <a Module with map 0x2ac0ae918319>#41#,__filename=0x25984467e849 <String[53]: /Users/james/code/canadiannessgit/spec/fbpspec.coffee>,__dirname=0x25984467e821 <String[38]: /Users/james/code/canadiannessgit/spec>)
   96: _compile [module.js:541] [pc=0x3f37ca73d2b0] (this=0x25984467e649 <a Module with map 0x2ac0ae918319>#41#,content=0x25984467ee01 <String[596]\: (function() {\n  var fbpspec, nodeRuntime;\n\n  fbpspec = require('fbp-spec');\n\n  nodeRuntime = {\n    label: "NoFlo node.js",\n    description: "",\n    type: "noflo",\n    protocol: "websocket",\n    secret: 'notasecret',\n    address: "ws://localhost:3333",\n    id: "7807f4d8-63e0-4a89-a577-2770c14f8106",\n    command: './node_modules/.bin/noflo-nodejs --verbose --debug  --catch-exceptions=false --secret notasecret --port=3333 --host=localhost --register=false --cache=true'\n  };\n\n  fbpspec.mocha.run(nodeRuntime, './spec', {\n    fixturetimeout: 20000,\n    starttimeout: 100000\n  });\n\n}).call(this);\n>,filename=0x25984467e849 <String[53]: /Users/james/code/canadiannessgit/spec/fbpspec.coffee>)
   97: .coffee [/Users/james/code/canadiannessgit/node_modules/noflo/node_modules/coffee-script/lib/coffee-script/register.js:16] [pc=0x3f37cab424b4] (this=0x3fd4c6006e99 <an Object with map 0x2ac0ae9d3109>#1#,module=0x25984467e649 <a Module with map 0x2ac0ae918319>#41#,filename=0x25984467e849 <String[53]: /Users/james/code/canadiannessgit/spec/fbpspec.coffee>)
   98: load [/Users/james/code/canadiannessgit/node_modules/noflo/node_modules/coffee-script/lib/coffee-script/register.js:45] [pc=0x3f37cab41f34] (this=0x25984467e649 <a Module with map 0x2ac0ae918319>#41#,filename=0x25984467e849 <String[53]: /Users/james/code/canadiannessgit/spec/fbpspec.coffee>)
   99: tryModuleLoad(aka tryModuleLoad) [module.js:417] [pc=0x3f37ca7342dd] (this=0x1d5407604189 <undefined>,module=0x25984467e649 <a Module with map 0x2ac0ae918319>#41#,filename=0x25984467e849 <String[53]: /Users/james/code/canadiannessgit/spec/fbpspec.coffee>)
  100: _load [module.js:409] [pc=0x3f37ca72fec2] (this=0x3fd4c6006f19 <JS Function Module (SharedFunctionInfo 0x22319c824c31)>#3#,request=0x25984467f2e1 <String[53]: /Users/james/code/canadiannessgit/spec/fbpspec.coffee>,parent=0x3448f427d7c1 <a Module with map 0x2ac0ae918319>#44#,isMain=0x1d5407604299 <false>)
  101: require [module.js:~465] [pc=0x3f37ca8191eb] (this=0x3448f427d7c1 <a Module with map 0x2ac0ae918319>#44#,path=0x25984467f2e1 <String[53]: /Users/james/code/canadiannessgit/spec/fbpspec.coffee>)
  102: require(aka require) [internal/module.js:20] [pc=0x3f37ca73e206] (this=0x1d5407604189 <undefined>,path=0x25984467f2e1 <String[53]: /Users/james/code/canadiannessgit/spec/fbpspec.coffee>)
  103: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/mocha/lib/mocha.js:220] [pc=0x3f37caa8a51a] (this=0x1d54076e4ef9 <JS Global Object>#5#,file=0x25984467f2e1 <String[53]: /Users/james/code/canadiannessgit/spec/fbpspec.coffee>)
  104: arguments adaptor frame: 3->1
  105: InnerArrayForEach(aka InnerArrayForEach) [native array.js:~946] [pc=0x3f37ca70d610] (this=0x1d5407604189 <undefined>,br=0x17b0b7069b51 <JS Function (SharedFunctionInfo 0x39efdb154d39)>#45#,bs=0x1d5407604189 <undefined>,w=0x17b0b7069bd9 <JS Array[2]>#46#,x=2)
  106: forEach [native array.js:~956] [pc=0x3f37ca70d38d] (this=0x17b0b7069bd9 <JS Array[2]>#46#,br=0x17b0b7069b51 <JS Function (SharedFunctionInfo 0x39efdb154d39)>#45#,bs=0x1d5407604189 <undefined>)
  107: arguments adaptor frame: 1->2
  108: loadFiles [/Users/james/code/canadiannessgit/node_modules/mocha/lib/mocha.js:217] [pc=0x3f37caa8a300] (this=0x17b0b7069bf9 <a Mocha with map 0x2ac0ae9d1169>#47#,fn=0x1d5407604189 <undefined>)
  109: arguments adaptor frame: 0->1
  110: run [/Users/james/code/canadiannessgit/node_modules/grunt-mocha-test/tasks/lib/MochaWrapper.js:51] [pc=0x3f37caa89ad1] (this=0x17b0b7069cf9 <a MochaWrapper with map 0x2ac0ae9d1219>#48#,callback=0x17b0b7069cb1 <JS Function (SharedFunctionInfo 0x39efdb14e7e1)>#49#)
  111: /* anonymous */(aka /* anonymous */) [/Users/james/code/canadiannessgit/node_modules/grunt-mocha-test/tasks/mocha-test.js:86] [pc=0x3f37caa855d0] (this=0x1d5407604189 <undefined>,complete=0x17b0b7069dd9 <JS Function (SharedFunctionInfo 0x39efdb14d8b9)>#50#)
  112: /* anonymous */(aka /* anonymous */) [/Users/james/code/canadiannessgit/node_modules/grunt-mocha-test/tasks/mocha-test.js:33] [pc=0x3f37caa84770] (this=0x1d5407604189 <undefined>,captureFile=0x1d5407604189 <undefined>,quiet=0x1d5407604189 <undefined>,run=0x17b0b7069d59 <JS Function (SharedFunctionInfo 0x39efdb14b331)>#51#,done=0x17b0b7069e69 <JS Function (SharedFunctionInfo 0x39efdb14b3f1)>#52#)
  113: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/grunt-mocha-test/tasks/mocha-test.js:81] [pc=0x3f37ca7eca8a] (this=0x17b0b7069f31 <an Object with map 0x2ac0ae9df291>#53#)
  114: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/grunt/lib/grunt/task.js:264] [pc=0x3f37ca97cbfa] (this=0x17b0b7069f31 <an Object with map 0x2ac0ae9df291>#53#,target=0x17b0b7069fa9 <String[6]: nodejs>)
  115: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/grunt/lib/grunt/task.js:82] [pc=0x3f37ca97a00e] (this=0x17b0b7069f31 <an Object with map 0x2ac0ae9df291>#53#,arg=0x17b0b7069fa9 <String[6]: nodejs>)
  116: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/grunt/lib/util/task.js:301] [pc=0x3f37ca979911] (this=0x17b0b7069f31 <an Object with map 0x2ac0ae9df291>#53#)
  117: runTaskFn [/Users/james/code/canadiannessgit/node_modules/grunt/lib/util/task.js:251] [pc=0x3f37ca979076] (this=0x11a67dadb801 <an Object with map 0x2ac0ae9d8439>#54#,context=0x17b0b7069f31 <an Object with map 0x2ac0ae9df291>#53#,fn=0x17b0b706a029 <JS Function (SharedFunctionInfo 0x3448f4207431)>#55#,done=0x212fdc833719 <JS BoundFunction bound  (BoundTargetFunction 0x212fdc833759)>#56#,asyncDone=0x1d5407604231 <true>)
  118: /* anonymous */ [/Users/james/code/canadiannessgit/node_modules/grunt/lib/util/task.js:300] [pc=0x3f37ca978da3] (this=0x11a67dadb801 <an Object with map 0x2ac0ae9d8439>#54#)
  119: arguments adaptor frame: 2->0
  120: /* anonymous */(aka /* anonymous */) [/Users/james/code/canadiannessgit/node_modules/grunt/lib/util/task.js:227] [pc=0x3f37ca97bdbf] (this=0x1d5407604189 <undefined>)
  121: _combinedTickCallback(aka _combinedTickCallback) [internal/process/next_tick.js:67] [pc=0x3f37ca97ba72] (this=0x1d5407604189 <undefined>,args=0x1d5407604189 <undefined>,callback=0x17b0b706a101 <JS Function (SharedFunctionInfo 0x3448f4207f19)>#57#)
  122: _tickCallback(aka _tickDomainCallback) [internal/process/next_tick.js:122] [pc=0x3f37ca97b6c7] (this=0x1d54076e0f49 <a process with map 0x2ac0ae91c0a1>#58#)
  123: /* anonymous */(aka /* anonymous */) [module.js:577] [pc=0x3f37ca72f9ed] (this=0x1d5407604189 <undefined>)
  124: run(aka run) [node.js:348] [pc=0x3f37ca72f808] (this=0x1d5407604189 <undefined>,entryFunction=0x22319c84b1e9 <JS Function Module.runMain (SharedFunctionInfo 0x22319c8259b1)>#59#)
  125: startup(aka startup) [node.js:140] [pc=0x3f37ca64054b] (this=0x1d5407604189 <undefined>)
  126: /* anonymous */(aka /* anonymous */) [node.js:463] [pc=0x3f37ca63e59f] (this=0x1d5407604101 <null>,process=0x1d54076e0f49 <a process with map 0x2ac0ae91c0a1>#58#)
=====================


==== C stack trace ===============================

 1: v8::Template::Set(v8::Local<v8::Name>, v8::Local<v8::Data>, v8::PropertyAttribute)
 2: SerialportPoller::Init(v8::Local<v8::Object>)
 3: init
 4: node::DLOpen(v8::FunctionCallbackInfo<v8::Value> const&)
 5: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&))
 6: v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::(anonymous namespace)::BuiltinArguments<(v8::internal::BuiltinExtraArguments)1>)
 7: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*)
 8: 0x3f37ca60961b
(node) v8::ObjectTemplate::Set() with non-primitive values is deprecated
(node) and will stop working in the next major release.

An in-range update of grunt-mocha-test is breaking the build 🚨

Version 0.13.3 of grunt-mocha-test just got published.

Branch	Build failing 🚨
Dependency	grunt-mocha-test
Current Version	0.13.2
Type	devDependency

This version is covered by your current version range and after updating it in your project the build failed.

As grunt-mocha-test is “only” a devDependency of this project it might not break production or downstream projects, but “only” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this 💪

Status Details

❌ continuous-integration/travis-ci/push The Travis CI build failed Details

Commits

The new version differs by 4 commits.

185d515 version 0.13.3
7966047 Fix failing timeout test
076c3ab Add clearCacheFilter option
8460539 old comment

See the full diff

Not sure how things should work exactly?

There is a collection of frequently asked questions and of course you may always ask my humans.

Your Greenkeeper Bot 🌴

Demo+docs for testing non-FBP systems

If one uses an existing FBP runtime with components in the fixture graph that drives the thing being tested, one can test anything with fbp-spec.

imgflo/imgflo-server#15 may be a good real-life example of this, where we are testing a node.js HTTP server.

Tests don't fail if component under test is not loadable

If component being tested is not loadable the tests don't fail in any way. They just start but neither the end nor completion callbacks get called, and there are no errors.

Simple example to duplicate this is NoFlo component with wrong datatype, like:

var noflo = require('noflo');

exports.getComponent = function() {
  var c = new noflo.Component();
  c.description = 'Multiplies numbers by arg';
  c.icon = 'cc';
  c.inPorts.add('in', {
    datatype: 'int',
    description: 'Number to multiply'
  });
  c.inPorts.add('by', {
    datatype: 'int',
    description: 'How much',
    required: true
  });
  c.outPorts.add('out', {
    datatype: 'all'
  });
  c.outPorts.add('error', {
    datatype: 'error'
  });
  noflo.helpers.WirePattern(c, {
    "in": ['in'],
    out: 'out',
    params: ['by'],
    forwardGroups: true,
    async: true
  }, function(data, groups, out, callback) {
    if (typeof data !== 'number') {
      return callback(new Error(data + ' is not a number'));
    }
    // do something with data
    // send output
    out.send(data * c.params.by);
    // tell WirePattern we are done
    return callback();
  });
  return c;
};

See the ERROR outport above, error is not a valid datatype in NoFlo.

Maybe support groups/brackets

Required to construct (sub)streams, which is a concept in FBP for structuring data, using IPs that mark 'openBracket' and 'closeBracket'.
This is sometimes used in NoFlo components/graphs.

Right now one has to use custom test components in the graph fixture for these.

Tool for adding testcase from flowtrace

Flowtrace can be captured of issues found in production. Want to be able to go quickly to a reproducable test-case - to understand, fix and ensure issue does not happen again.

A tool could allow specifying a flowtrace, and an an fbp-spec file (with already existing topic/fixture/testcases). It would read the test topic, then go through the trace to find the inputs/output to that component. Then create a testcase with this input and skeleton/empty expectations, and modify the YAML file to include the new testcase.

Tool would be used (primarily) manually by developers. But one can imagine automated uses, where a production failure would automatically create testcase, put it through CI against newest/development version of the software.

Ctrl+C / SIGINT should quit process cleanly

I have been getting familiar with fbp-spec and I have noticed that for some reason the process doesn't exit properly. It just hangs and my only option is to killall node.

It looks like something is hijacking SIGINT in fbp-spec (at least on macOS 10.12) because I can't CTRL+C out of the process either.

Same problem happens with noflo-nodejs.

edit This could be related to error handling as outlined in #137 because I noticed that my test had a broken fixture syntax thanks to auto-indent so now it works and exits properly.

An in-range update of fbp-protocol-client is breaking the build 🚨

Version 0.1.12 of fbp-protocol-client just got published.

Branch	Build failing 🚨
Dependency	fbp-protocol-client
Current Version	0.1.11
Type	dependency

This version is covered by your current version range and after updating it in your project the build failed.

fbp-protocol-client is a direct dependency of this project this is very likely breaking your project right now. If other packages depend on you it’s very likely also breaking them.
I recommend you give this issue a very high priority. I’m sure you can resolve this 💪

Status Details

❌ continuous-integration/travis-ci/push The Travis CI build failed Details

Commits

The new version differs by 8 commits.

a0930d5 Bump
665dd92 Travis: Skip cleanup
d258b00 Merge pull request #108 from flowbased/connection-errors
f74304e NodeWebSocketClient: Use correct event name for connection errors
792874d Merge pull request #107 from flowbased/no_coffeescript_runtime
d19a7b3 Ignore build artifacts
e7b01ba Remove obsolete component.io manifest
b7dc9f1 Remove CoffeeScript as a runtime dependency

See the full diff

Not sure how things should work exactly?

There is a collection of frequently asked questions and of course you may always ask my humans.

Your Greenkeeper Bot 🌴

Sending to multiple input ports

I saw in the Readme

You can send data to multiple inports and check expectations on multiple ports per testcase:

I must admit that I don't really understand what I assume is an example of this (below the quoted line), but it sounds very different from "classical" FBP. So my question is whether there are any plans to accommodate "classical" FBP in fbp-spec, esp. as the preamble says:

A runtime-independent test framework for Flow Based Programming (FBP) component and graphs

TIA

Loading tests with https fails

Seems to think its a local file instead

An in-range update of fbp is breaking the build 🚨

Version 1.7.0 of fbp was just published.

Branch	Build failing 🚨
Dependency	fbp
Current Version	1.6.0
Type	dependency

This version is covered by your current version range and after updating it in your project the build failed.

fbp is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details

❌ continuous-integration/travis-ci/push The Travis CI build failed Details

Commits

The new version differs by 23 commits.

4a87a60 Release 1.7.0
eac460d Merge pull request #88 from flowbased/annotations
dbb4660 Move newline matching to a named rule
9f15f3d Include node names in data validation errors
0ce2fcd Clean up tests and ensure node names are there
65214c1 Fix FBP serialization in case sensitive mode
bd28c52 Document annotations
c629173 Add test to verify that multiple arrayport connections on same line are fine, fixes #19
2219408 Allow whitespace in annotation values
8d80841 Move prop check to separate test case
86e3d3b Check properties with roundtrip
55120c7 Add parsing and serialization for annotations
0944d2d We now provide default values
265476d Name is inside props
43e5797 Export textual properties as annotations

There are 23 commits in total.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.

Your Greenkeeper Bot 🌴

Network doesn't get stopped after tests have run

Now when running fbp-spec tests, the network is not stopped. This may be (at least partial) cause for #8.

Can be seen when running specs with Flowhub, for example:

(at the time of screenshot all tests had been run)

Missing timeouts for protocol operations

Either in runner.coffee or in protocol.coffee. For startNetwork, stopNetwork, sendPacket etc.
Together with #11, probably should be put in a general facility to ensure consistent behavior.

runner must respect the timeouts set in the spec files, and report them.

Handle network errors

Currently fbp-spec just hangs if there is a network error, in this case related to starting the fixture graph:

 fbp-spec:runner:received network error { message: 'Component CreateDataType not available with base /home/bergie/Projects/big-iot/bigiot-bridge',

Related to #72

Schemas not available in browser/component.io

Either we need to figure out a way to include the .YAML schemas in the component.io build,
or need to commit the build .json files into the git repo.

Third option is to get rid of the component.io shit

Compatibilty/legacy support for Mocha testcases

Right now we have, across component libraries and projects, many existing tests. Long-term we'd like to migrate the majority of these to fbp-spec, but this is not done in a day....

Most of these tests are written in Mocha+Chai. They usually either set up a NoFlo component by requiring it directly, or use Noflo.ComponentLoader (especially with sub/graphs). Component lifetime is sometimes done per testcase, and sometimes for the whole suite.
We also have some Mocha+Chai tests, which test on the MsgFlo participant level. And we sometimes have tests for plain (JavaScript) code used by NoFlo component, but which is not NoFlo?FBP...

To enable an easy, immediately useful and gradual migration to fbp-spec. It would be great if we could have a way to represent/enumerate, run and evalute plain-old-MochaChai tests as if they were fbp-spec.

Schemas not available in browser/component.io

Either we need to figure out a way to include the .YAML schemas in the component.io build,
or need to commit the build .json files into the git repo.

Third option is to get rid of the component.io shit

No error on permission problems

To reproduce:

Run noflo-nodejs --register=false without --secret set
Run fbp-spec against it

Expected: Clear error explaining the permissions issue
Actual: Hangs forever without any output or warning

Related to #11

Stops on first failing test

If you have two test cases and the first one fails, then the second test doesn't get run.

Before/After graphs

In addition to the actual test fixture, it might be useful to have some before and after graphs for dealing with preparing/tearing down state for components dealing with side effects.

Inconsistent results when run inside Flowhub

Here is how Flowhub triggers fbp-spec (with a pre-existing runtime connection): https://github.com/noflo/noflo-ui/blob/master/components/RunTests.coffee

However, the results are inconsistent when shown in the UI:

This doesn't conform to what I see in the runtime console where the states appear to be what was expected:

undefined 'Portal c-base space station is L4 RESISTANCE'
undefined 'Portal c-base space station is L8 RESISTANCE'
undefined 'Portal c-base space station is L8 ALIENS'
undefined 'Portal c-base space station (NEUTRAL) is L8 and has disco'
undefined 'Portal c-base space station is L8 RESISTANCE'
undefined 'Portal c-base space station has special disco mods'
undefined 'Portal c-base space station has special disco mods’

When the same tests are run inside Node.js Mocha runner, they all pass: https://travis-ci.org/c-base/ingress-table/builds/77374093#L446

Don't fail if cannot get component source

In MsgFlo for instance, sometimes the component source code is unknown (if component is declared directly, not via a handler). Right now that causes fbp-spec to fail, we try to get the source to find runtime-provided tests. But these are not absolutely needed, so should continue without source for a component if not returned. Maybe emit a warning

IIPs of fixture graph never get sent

We should make sure the fixture graph gets started, including IIPs being sent before the input packets. Otherwise one has to resort to hacks like:

noflo/noflo-hackrf@20112de

An in-range update of mocha is breaking the build 🚨

Version 3.5.1 of mocha just got published.

Branch	Build failing 🚨
Dependency	mocha
Current Version	3.5.0
Type	dependency

This version is covered by your current version range and after updating it in your project the build failed.

mocha is a direct dependency of this project this is very likely breaking your project right now. If other packages depend on you it’s very likely also breaking them.
I recommend you give this issue a very high priority. I’m sure you can resolve this 💪

Status Details

❌ continuous-integration/travis-ci/push The Travis CI build failed Details

Release Notes

v3.5.1

3.5.1 / 2017-09-09

📰 News

📣 Mocha is now sponsoring PDXNode! If you're in the Portland area, come check out the monthly talks and hack nights!

🐛 Fixes

#2997: Fix missing xit export for "require" interface (@solodynamo)
#2957: Fix unicode character handling in XUnit reporter failures (@jkrems)

🔩 Other

#2986: Add issue and PR templates (@kungapal)
#2918: Drop bash dependency for glob-related tests (@ScottFreeCode)
#2922: Improve --compilers coverage (@ScottFreeCode)
#2981: Fix tpyos and spelling errors (@jsoref)

Commits

The new version differs by 14 commits.

4070a44 Release v3.5.1
466ba73 update CHANGELOG.md for v3.5.1 [ci skip]
1cc0fc0 import/require xit, fixes #2972
74fa66f update nyc to latest; remove workaround in travis script
aa52933 update coveralls strategy; closes #2984
73a5338 Spelling (#2981)
9f403bf Add utils.escape tests and fix unicode escaping
800acbc whitelist "developer-experience" tag for stalebot [ci skip]
5895671 Added issue, pull request templates. (#2869)
075bd51 Merge pull request #2918 from mochajs/no-shell-test
8710438 Work around Node 0.10 Windows flake when testing
13b8340 Ensure that compiler lookup works and not just that transpilation works (#2922)
26d337a Add tests for double-star behavior
c0e6b68 Eliminate glob.sh

See the full diff

Not sure how things should work exactly?

There is a collection of frequently asked questions and of course you may always ask my humans.

Your Greenkeeper Bot 🌴

Hangs forever if command does not write to stdout

It is used as a proxy for whether the runtime is ready to accept connections. Which is very hacky.
Instead we should periodically try to open the runtime connection, and if it does not succeed with a certain time, fail with an explanatory error

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

flowbased / fbp-spec Goto Github PK

fbp-spec's Introduction

fbp-spec

Status:

Purpose & Scope

License

Usage

Installing

Writing tests

Multiple ports

Sequence of packets

Extract data using path

Skipping tests

Using fixtures

Supported assertions

More

Running tests with fbp-spec commandline tool

Running tests by integrating with Mocha

Running tests interactively in Flowhub

Generating tests programatically

Integrating test runner in an application

Add supporting for a new runtime

Writing a test runner in another language

fbp-spec's People

Contributors

Stargazers

Watchers

Forkers

fbp-spec's Issues

Version 0.1.14 of fbp-protocol-client was just published.

Version 1.12.6 of coffee-script just got published.

Version 3.5.1 of bluebird just got published.

Version 0.13.3 of grunt-mocha-test just got published.

Version 0.1.12 of fbp-protocol-client just got published.

Version 1.7.0 of fbp was just published.

Version 3.5.1 of mocha just got published.

3.5.1 / 2017-09-09

📰 News

🐛 Fixes

🔩 Other

Recommend Projects

Recommend Topics

Recommend Org