nanoporetech / data-rambler Goto Github PK
View Code? Open in Web Editor NEWAn experimental language for a JSON query, transformation and streaming
License: Mozilla Public License 2.0
An experimental language for a JSON query, transformation and streaming
License: Mozilla Public License 2.0
Using the JSONata playground example (see below) I get different output when using JSONata
and Data Rambler
With the expression Account.Order.[OrderID, [Product.'Product Name']]
JSONata gives:
[
[
"order103",
[
"Bowler Hat",
"Trilby hat"
]
],
[
"order104",
[
"Bowler Hat",
"Cloak"
]
]
]
However Data Rambler gives flattening the result
:
[
"order103",
"Bowler Hat",
"Trilby hat",
"order104",
"Bowler Hat",
"Cloak"
]
The expected output from the expression is that of the JSONata output above
Input source:
{
"Account": {
"Account Name": "Firefly",
"Order": [
{
"OrderID": "order103",
"Product": [
{
"Product Name": "Bowler Hat",
"ProductID": 858383,
"SKU": "0406654608",
"Description": {
"Colour": "Purple",
"Width": 300,
"Height": 200,
"Depth": 210,
"Weight": 0.75
},
"Price": 34.45,
"Quantity": 2
},
{
"Product Name": "Trilby hat",
"ProductID": 858236,
"SKU": "0406634348",
"Description": {
"Colour": "Orange",
"Width": 300,
"Height": 200,
"Depth": 210,
"Weight": 0.6
},
"Price": 21.67,
"Quantity": 1
}
]
},
{
"OrderID": "order104",
"Product": [
{
"Product Name": "Bowler Hat",
"ProductID": 858383,
"SKU": "040657863",
"Description": {
"Colour": "Purple",
"Width": 300,
"Height": 200,
"Depth": 210,
"Weight": 0.75
},
"Price": 34.45,
"Quantity": 4
},
{
"ProductID": 345664,
"SKU": "0406654603",
"Product Name": "Cloak",
"Description": {
"Colour": "Black",
"Width": 30,
"Height": 20,
"Depth": 210,
"Weight": 2
},
"Price": 107.99,
"Quantity": 1
}
]
}
]
}
}
We could support regex literals in the scanning stage, this would allow us to process strings in much more powerful ways and would be consistent with JS behaviour. Supporting these in the scanner would be tricky, but not impossible.
Path expressions can include segments that sort the incoming data. We support parsing these but do not evaluate them at this time. They are slightly complicated in that they support sorting by multiple criteria, so this will require a slightly more complex implementation and careful consideration of JSONata behaviour.
It's not clear at this time if sorting occurs over an entire match of a path expression or just on the current branch ( like filters ).
Path expressions use an automatic packing/unpacking algorithm in several places. It works on 2 equivalency rules, where any place one would be accepted the other must also. Also when resolving a value the latter is the preferred output.
[ value ] == value
[] == undefined
This means in some scenarios you will get a value or undefined instead of an array when the input data changes, making the expressions somewhat unpredictable.
I propose we modify this behaviour slightly to make it more predictable. We unfortunately cannot do this everywhere as it breaks some nice to haves.
let output = ({ a: 1 }).a
Our current path expression evaluator is based on a breadth first process that processes data for each path segment in sequence. This is a relatively optimal way to evaluate content but it comes with some tradeoffs in compatibility.
The original algorithm in the JSONata spec is based on a recursive depth first system. This is well suited for certain concepts like binding a variable for a particular branch of the tree. To work around this in our algorithm we had to add a special recurse step for this variable binding, but this doesn't work perfectly.
Resolving these edge cases is becoming increasingly complex and fragile, we've reached the point where the easiest way forward would be to replace our existing algorithm with a depth first system.
The following functions from JSONata are missing.
In some cases we have provided syntactic capability that replaces them, in others it's not clear that they really need to be part of the stdlib. But there are still a number which still need to be implemented.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.