Comments (11)
Hi Martin. Thanks for raising this. At the moment I am struggling to understand exactly the issue that are describing, so if you could expand with code examples, that would be really useful.
Iām wondering if this is an issue with the jsonata language itself, or with the implementation. JSONata is designed as a declarative language and not an imperative language. In other words, the syntax is used to describe the desired result and not the steps a computer has to take to achieve that result. So there is no syntax to control the flow of execution. I only say that because that new syntax you have proposed looks as if its purpose is to control the flow of execution. I also need to mindful of not tying jsonata into only being implementable in javascript. I know of a group who are creating a native implementation in Java.
Having said all that, Iām keen to understand the issue further so that we can enable your use-case.
from jsonata.
Hello Andrew.
Jsonata becomes a very versatile general purpose tool with these small additions. So moving a little bit outside of the philosophy could reap huge benefits. I'm using this on a custom platform almost identical to node-red you have at IBM.
I see jsonata as a json object manipulation tool. With 3 lines of code i could extend to also accept data from a database via an orm. So in fact, I see my database as one gigantic json blob.... Perfect for jsonata. I believe we don't need much imagination to see the power in that. However in some cases then the performance is not that good and being able to use the power of the orm is critical.
if you want to build a sql string with an orm, you do something like: meta.build().select("*").from("table") and get an output: "SELECT * FROM "table"". Then at the end of these chained methods you wait for the promise. Now to get that to work, I need to tell jsonata when to wait and when not to. In this case after all functions. In another case however, say: "$meta($first().second)" I would like to retrieve the result after each method, thereby not waiting until the end.
Jsonata does the first by standard and the second in my extension, I need both.....
Hope this makes sense.
Cheers Martin
from jsonata.
Sorry one more thing. The first case works perfectly well with vanilla jsonata, so the "imperative" part works very well. It is when I want to mix, I have problems.
from jsonata.
The problem here, I think, is that the knex Builder
object which chains all the select/from/etc methods is also a thenable
object. When jsonata sees that, it always tries to invoke the then
function as part of its async processing. However, for knex, we only want this to happen at the end of the function chain. I have created a fix for this in branch thenable together with a set of unit tests that invoke a trivial example of this 'chainable/thenable' pattern. I have also installed and tested with with knex to verify that expression such as $knex.select().from("authors").where("isbn", "=", $knex.select().from("books").isbn)
work as expected.
Let me know if this works for you.
from jsonata.
First of all, tremendous thanks for building this addition, it is of great value here!:-)
I made my own tests and confirm that it works as defined in the unit tests.
So now we are almost there, there is only one small piece missing and that is to wait for the promise inside the expression. I see that using the standard $count function it does wait for the promise to fulfill before the count is called but this does not seems to work for custom functions...
Example:
( $vari := $count($knex("business_document").select().where({"id":"1"})); {"return":$vari} )
works as the result becomes {return:1}
. However with a custom dummy function I get:
( $vari := $dummy($knex("business_document").select().where({"id":"1"})); {"return":$vari} )
{return:Builder}
. That is, the knex builder and not the result.
Is there any way to add this functionality also to custom functions? Or add a dummy function to the standard ones?
Again many, many thanks for building this. It makes jsonata even more awsome!
from jsonata.
Hmm, that's odd. The following code works for me, could you try running this, please:
var jsonata = require('jsonata');
var knex = require('knex')({
client: 'sqlite3',
connection: {
filename: "./mydb.sqlite"
},
useNullAsDefault: true
});
expression = '( $vari := $dummy($knex("business_document").select().where({"id":"1"})); {"return":$vari} )';
var dummy = function(arg) {
console.log('DUMMY', arg);
return arg;
};
var bindings = {
'knex': knex,
'dummy': dummy
};
var expr = jsonata(expression);
var callback = function(err, result) {
if(err) {
console.log('ERROR:', err);
} else {
console.log('Result:', result);
}
};
expr.evaluate({}, bindings, callback);
I get the following output:
DUMMY { id: '1', surname: 'coleman', isbn: '0123456789' }
Result: { return: { id: '1', surname: 'coleman', isbn: '0123456789' } }
so the value of arg
passed to custom function dummy
is the result from the DB call. I'd be interested in seeing your code.
from jsonata.
The difference seems to be that I use async/await.
With a callback as in your example above, I get the same result as you. With the following:
var result = await expression.evaluate({}, bindings);
I get the builder in the result.
Can I conclude that I should avoid using async/await and wrap my callback instead? Could this be made to work with async/await?
Many, many thanks, Martin
from jsonata.
Hi Martin,
Yes, you do need to supply the callback argument to the evaluate method. This switches the engine into a mode that supports asynchronous functions (custom functions that return promises). Without this callback, the engine reverts to a pure synchronous mode (for backwards compatibility reasons prior to v1.2). As you say, you can wrap the callback instead, as I've done in the unit tests:
var jsonataPromise = function(expr, data, bindings) {
return new Promise(function(resolve, reject) {
expr.evaluate(data, bindings, function(error, response) {
if(error) reject(error);
resolve(response);
});
});
};
Given you now have this working, I'll merge thenable
into master
, and publish a minor release to NPM.
from jsonata.
Hello Andrew,
Thank you so much, this is a wonderful feature!
Speaking about merging, when will the 1.4 be merged as well? That is another extremely useful feature what is tremendously useful for us.
Many thanks, Martin
from jsonata.
Hello Andrew,
Today I tested with chained method with binding and then it did not go so great:
var bindings = {
'knex': knex,
'dummy': dummy
};
var expression = jsonata(`$knex($.table).select().where({"last_name":$.name})`);
expression.evaluate({table:"contract_party", name:"Dumont"},bindings,(err,res)=>{
console.log(err);
console.log(res);
});
In this case table input (contract_party) is picked up correctly but name (Dumont) is evaluated to undefined in the where method.
UPDATE:
This can be solved by using $$ instead of $ as context:
var expression = jsonata(`$knex($.table).select().where({"last_name":$$.name})`);
Note that some times when I run the above, the jsonata inside the where method does not get executed, strange error I cannot reproduce anymore.
Cheers
Martin
from jsonata.
Closing this issue as resolved. Feel free to re-open if you have any more problems with this.
from jsonata.
Related Issues (20)
- How to return the value from jsonata instead of promise/async HOT 3
- Dry run, return expected schema for a given input schema
- How to handle undefined value HOT 2
- $formatDateTime(millis, picture [, timezone]) and $parseDateTime(timestamp, picture) HOT 1
- Clarification : Is there Any Feature for Apply Break or Continue in Looping HOT 1
- parallel map
- switch case or if else
- Truthiness of numbers is completely random
- Help resolve current pinning to 1.8.6 for JSONata-Wrapper (possibly ducktape compatibility)
- Reserved Tokens Documentation
- ($.some_non_existent_key != 'anything') evaluates to false HOT 3
- $string(number) capped at precision-15
- Error handling and asynchronous behavior in evaluate function
- Rounding number without "Round half to even"
- There isn't a uuid() function for JSONata HOT 2
- Hard spaces in documentation not rendered correctly
- Security contact for JSONata HOT 4
- default query on try.jsonata.org HOT 1
- is 2.0.4 a fake release? HOT 4
- Multi-level JSON processing: Unwrapping Objects
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
š Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ššš
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ā¤ļø Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from jsonata.