Consider exposing interface for repeated execution with varying environment about go-jsonnet HOT 11 CLOSED

I like the interface you proposed. I was thinking about something similar in the context of #82 (but there the idea was to reuse the whole interpreter or parsed std object with different programs).

from go-jsonnet.

I'm sure I'm making some rookie mistake here, failing to take the late-bound processing requirements into account, and I don't want to propose something that limits how the language works.

I came to this idea after fiddling around with the jsonnet command-line tool for a while, where I was feeding in various JSON files via the --tla-code-file flag to the same Jsonnet expression (starting with function(...)) repeatedly. I'm not sure whether using the --tla-str-file flag would have worked just as well, given that the input content was JSON, and not Jsonnet per se.

from go-jsonnet.

I'm sure I'm making some rookie mistake here, failing to take the late-bound processing requirements into account, and I don't want to propose something that limits how the language works.

I don't see any problem, what do you mean? I like the proposal, I think it would work. You can parse and even evaluate the jsonnet function just once and then call it multiple times with different arguments, no problem.

I'm not sure whether using the --tla-str-file flag would have worked just as well, given that the input content was JSON, and not Jsonnet per se.

A string containing the file contents would be passed.

from go-jsonnet.

I don't see any problem, what do you mean?

I figured that we wouldn't want to claim that the snippet had been compiled if it makes use of some imported code that wasn't known to the environment at compilation time, but that could be provided later via --ext-code or --ext-code-file. I'm showing my lack of knowledge of Jsonnet here, but I thought there are some tricks you can play where the code supplied as the primary snippet is incomplete.

You can parse and even evaluate the jsonnet function just once and then call it multiple times with different arguments, no problem.

There, what is the distinction between parsing and evaluating? It sounds like evaluation happens before the arguments are available.

from go-jsonnet.

what is the distinction between parsing and evaluating? It sounds like evaluation happens before the arguments are available.

Short: yes, evaluation happens before arguments are available. The evaluation of the function definition to a function value. Function value can then be called with arguments. You can see how it works with TLA in a work in progress PR here: https://github.com/google/go-jsonnet/pull/110/files#diff-08a04131bb8a1427caf90a8f256938a6R828

Parsing goes from source code to abstract syntax tree (AST) - a structured representation of the source.

Evaluation starts with the AST and the whole execution environment (extVars, importer, etc.) and produces a value. (Often, like in the case of vm.EvaluateSnippet it actually starts with the source code parsing is treated as the part of evaluation).

So for example parsing of 2 + 2 results in BinaryOperator{op: "+", left: NumberLiteral{2}, right: NumberLiteral{2}} and evaluation of this expression produces 4.

The same applies to functions, i.e. parsing produces AST node that represents a lambda expression (function definition) and evaluation would result in a function value which can be actually called.

So for example you can have an expression like:

local foo(x) = x, bar(x) = 42; foo

This snippet will be parsed to something like:

Local{
    binds: {
        foo: FunctionDefinition{params: ["x"], body: Var{"x"}},
        bar: FunctionDefinition{params: ["x"], body: NumberLiteral{42}},
    },
    body: { Var{"bar"} }
}

While evaluation result is a function value - something that can be called with an argument and it would produce a value 42.

Another example:

std.parseInt

The AST looks like Index{ target: Var{"std"}, index: "parseInt" }. It evaluates to a function value that can be called with a string argument and it would produce an int value.

(Note: The actual AST nodes are more complex, simplified for clarity).

I figured that we wouldn't want to claim that the snippet had been compiled if it makes use of some imported code that wasn't known to the environment at compilation time, but that could be provided later via --ext-code or --ext-code-file.

There are 3 ways to pass values to jsonnet:

• imports
• external variables
• top level arguments

None of them are necessary to parse jsonnet. Parsing just takes the source code and produces AST. We have a function that does exactly that: func snippetToAST(filename string, snippet string) (ast.Node, error).

To evaluate jsonnet you need imports and external variables set up (because they can be referred to from anywhere in the code). The result of evaluation is a jsonnet value. If you want to change imports or external variables, you need to evaluate it again.

If the result of evaluation was a function and we have some top-level args, we can call the function with these top level arguments. You can see how it can work here: https://github.com/google/go-jsonnet/pull/110/files#diff-08a04131bb8a1427caf90a8f256938a6R828

from go-jsonnet.

That is very helpful to read. Thank you for explaining in such detail.
I'll go through more of the code to play along.

from go-jsonnet.

If the result of evaluation was a function and we have some top-level args, we can call the function with these top level arguments.

And yes, that's exactly what I'm interested in getting a handle on for my use case. However, it sounds like my original interface doesn't correctly capture the constraint that its utility would be for Jsonnet code that produces a function.

If I called the hypothetical (*vm).PrepareSnippet to yield a *Program, I wouldn't want to get back just a parsed AST that would become a function when later evaluated, which could then be called; instead, I'd want a *Program to be the result of evaluation against whatever environment had been available as of the call to (*vm).PrepareSnippet, as you had confirmed earlier.

What happens, though, if the code snippet passed to (*vm).PrepareSnippet doesn't yield a function? Would it be an object already manifested as JSON text that's just held in the Program instance, copied out for each call to (*Program).Eval? Would it be an object that gets manifested on each call to (*Program).Eval?

Alternately, if the function was called (*vm).PrepareFunction, it could return an error if the snippet didn't yield a function. I'm not sure it's necessary to be that strict about it.

from go-jsonnet.

What happens, though, if the code snippet passed to (*vm).PrepareSnippet doesn't yield a function?

Current behavior is that if you specify TLAs and the result is not a function, TLAs get ignored (in both C++ and Go). But I think it should return an error.

Would be an object already manifested as JSON text that's just held in the Program instance, copied out for each call to (*Program).Eval?

I think if we're keeping function value when it's a function, we should keep an object value if it's an object.

Would it be an object that gets manifested on each call to (*Program).Eval?

Yes, that would be reasonable.

Alternately, if the function was called (*vm).PrepareFunction, it could return an error if the snippet didn't yield a function. I'm not sure it's necessary to be that strict about it.

I think that's the best option. It doesn't really make sense to evaluate in two stages for objects.

from go-jsonnet.

Wondering what's going on with this issue.

I was expecting something like VM::EvaluateParsedSnippet(node ast.Node).. but only found functions that would parse the node again..

Would it make sense to implement such a function?!

from go-jsonnet.

I'm also wondering what's the use of SnippetToAST if no function take the ast.Node as an input?! :)

from go-jsonnet.

Fixed in #318.

from go-jsonnet.