It's going to be useful for development of tools.

For now it's possible to have a generic dump using go-spew (https://github.com/davecgh/go-spew).

$ go-jsonnet --max-stack 7 -e 'local f(n, c=0) = if n == 0 then c else f(n - 1, c + n) tailstrict; f(100)'
RUNTIME ERROR: Max stack frames exceeded.
        <std>:979:29-30 thunk from <thunk <ta> from <function <anonymous>>>
        <builtin>       builtin function <type>
        <std>:981:33-35 thunk from <function <anonymous>>
        <builtin>       builtin function <primitiveEquals>
        <std>:981:12-40 function <anonymous>

        <cmdline>:1:69-75       $
        During evaluation

When fixed, remove special go golden for test_cmd case max_trace4

By avoiding parsing the std every single time.

First we need to have some benchmarking in place, and some reasonable "corpus" of jsonnet to test on. We can start by using current test suite, but dedicated benchmarks make more sense.

Once we have that, we can try to find bottlenecks and potentially optimize. Instead of having them scattered throughout the code, I'll dump some of these ideas here:

• caching of object locals (they're desugared to locals in each field now)
• caching of object fields (controversial and important, we need a discussion first)
• memory usage of tail recursion
• making sure that we don't keep objects for too long
• generate unique numbers for variables statically and use that instead of names
• faster implementation of std.join, uniq, setInter and other array building builtins
• native implementation of std.sort, std.parseInt and other functions that the user may expect to be optimized
• interning of booleans for faster comparisons (by using the pointer directly and never actually dereferencing) and lower memusage.
• avoid desugaring operators, call builtins directly instead (fewer things to check, no need to do std lookup and verifying that it's actually a function etc.).
• fast & easy way for users to build arrays- perhaps a more sophisticated representation of an array would help, so that + behaves reasonably and/or more native higher-order functions. This stuff often applies to strings, too.
• faster indexing of objects with deep inheritance chains
• reduce overhead of function call argument checking (currently we check for a lot of special cases, because of optional arguments and that requires building maps and multiple lookups for each argument - maybe it would help to skip these checks when there are only positional arguments)
• check performance of environment capturing (and reuse the environment when possible)
• interning of identifiers
• try other representations of cachedThunk

Before we implement any of these we should have a benchmark that proves that it actually helps.

While the RuntimeError type is exported, there does not seem to be an exported way to construct a RuntimeError or an exported way to populate the StackTrace ([]TraceFrame) from.

It would be useful to be able to have an exported variant of the get* functions in evaluator.go that return Go types (e.g. string instead of valueString) and a RuntimeError. However, just being able to get the stacktrace and exporting makeRuntimeError would be sufficient.

Returning other error types from a native function results in an "INTERNAL ERROR" error message, requesting that the user file a bug. The lack of a stack trace in the error message can make it difficult to know what caused the error.

Still there is a bunch of builtins that are available in C++ version and not here.

Current status:
builtins["makeArray"] = &Interpreter::builtinMakeArray; DONE
builtins["pow"] = &Interpreter::builtinPow; DONE
builtins["floor"] = &Interpreter::builtinFloor; DONE
builtins["ceil"] = &Interpreter::builtinCeil; DONE
builtins["sqrt"] = &Interpreter::builtinSqrt; DONE
builtins["sin"] = &Interpreter::builtinSin; DONE
builtins["cos"] = &Interpreter::builtinCos; DONE
builtins["tan"] = &Interpreter::builtinTan; DONE
builtins["asin"] = &Interpreter::builtinAsin; DONE
builtins["acos"] = &Interpreter::builtinAcos; DONE
builtins["atan"] = &Interpreter::builtinAtan; DONE
builtins["type"] = &Interpreter::builtinType; DONE
builtins["filter"] = &Interpreter::builtinFilter; DONE
builtins["objectHasEx"] = &Interpreter::builtinObjectHasEx; DONE
builtins["length"] = &Interpreter::builtinLength; DONE
builtins["objectFieldsEx"] = &Interpreter::builtinObjectFieldsEx; DONE
builtins["codepoint"] = &Interpreter::builtinCodepoint; DONE
builtins["char"] = &Interpreter::builtinChar; DONE
builtins["log"] = &Interpreter::builtinLog; DONE
builtins["exp"] = &Interpreter::builtinExp; DONE
builtins["mantissa"] = &Interpreter::builtinMantissa; DONE
builtins["exponent"] = &Interpreter::builtinExponent; DONE
builtins["modulo"] = &Interpreter::builtinModulo; DONE
builtins["extVar"] = &Interpreter::builtinExtVar; DONE
builtins["primitiveEquals"] = &Interpreter::builtinPrimitiveEquals; DONE
builtins["native"] = &Interpreter::builtinNative; TODO: see #44
builtins["md5"] = &Interpreter::builtinMd5; DONE

Hi jsonnet team,

As golang is now the preferred language for ops, will go-jsonnet become the de facto implementation for jsonnet?

IMHO, the go version is more likely to be adopted by the community as a library. Also from code readability perspective, golang version is more likely to get merge-able PRs.

If I were writing a jsonnet formatter, I'd like to have access to comments as AST nodes so I can include them when I'm rewriting the source. It would also be helpful if using the AST to generate jsonnet as a conversion from another source.

At present, it appears that go-jsonnet's public interface starts with jsonnet.MakeVM, against which one can call (*VM).ExtVar, (*VM).ExtCode, and (*VM).Importer to prepare an environment, and eventually call (*VM).EvaluateSnippet against a sequence of Jsonnet expressions.

That call to (*VM).EvaluateSnippet eventually calls on the internal snippetToAST function, which yields a parsed ast.Node ready for later use in the evaluate function. In there, we find some more code-to-AST translation for environment entries that are Jsonnet code.

Is it viable to cleave a new break into this interface, so that it's possible to parse some Jsonnet—together with its import callbacks, and perhaps code entries from the environment—and yield a prepared object that could then be used for repeated evaluation against different TLAs?

The use case I have in mind is a server that would take a Jsonnet function that expects to receive some JSON as a TLA and evaluate the same Jsonnet function repeatedly and frequently, just with its input changing. It's unfortunate to have to restart this procedure each time from raw text when the code involved won't change between evaluations, throwing away the parsed form of the input text.

I concede that I have no measurements to critique the cost of the current interface compared to the separate prepare-once/evaluate-repeatedly approach sketched above. Has this project considered and rejected this approach before? If not, I'm willing to help pursue it.

A few ideas for an interface:

• (*vm).PrepareSnippet(filename string, snippet string) (*Program, error)
• (*Program).Eval(vs map[string]string) (string, error)
• (*Program).EvalTo(w io.Writer, vs map[string]string) error

We want to be able to use go version as a drop-in replacement for C++ version (for evaluation only, no formatting). This will probably require some changes to happen first on C++ side, e.g. separation of evaluation and formatting.

It's a part of the larger API discussion.

@mqliang wrote in #89:

It would be very interesting if we can dump ast as go source files, then we can dump ast of stdlib as go source code, and embed it into vm, thus avoid parsing stdlib every time. Any idea?

I think we should look into it. It sound quite easy.

Some constructs are desugared to functions from std. The problem is that if the user wraps them in their own definition of std they can change the meaning of core language constructs (probably just break them) which will be extremely confusing.

We can have special alias for std, unavailable for the user, for example $td.

They are currently ignored

The parameter names are visible to the user - they can use named arguments instead of positional for every function, including those in stdlib and builtins. Generally the names of parameters should be considered a part of the interface.

So we need to make sure they are right (consistent with http://jsonnet.org/docs/stdlib.html) and it should be verified by tests.

We're affected by this: golang/go#14932

BTW it seems it is non-trivial to have exactly the same values of numeric functions on all platforms - golang/go#20319.

Some constructs using super don't work yet properly.

It would be nice to use go version as a commandline drop-in replacement for C++ version.

We can avoid implementing it in Go for now if we implement libjsonnet #46 and use that from C++ commandline. The advantage is full compatibility, more tests of go-jsonnet (currently libjsonnet tests are very minimal, interesting tests happen through invoking jsonnet command) and we get it almost for free once we have #46.

I've set myself an arbitrary objective today to allow passing through JSON, i.e. add support for desugaring, checking, executing, manifesting basic objects and arrays containing other objects / arrays / primitives.

Accessing a field multiple times results in multiple evaluations. This can be very easily avoided and innocent looking code may be in fact exponential.

I know there are some concerns about memory usage.

https://imgur.com/a/87nf9

Expected and actual output seem to be randomly mixed together. Maybe line-based diff would be better after all.

Super can be a perfectly good value. It's not an object, though. It would be just a selfBinding - it's indexable, but + doesn't make sense on it.

We could desugar in super etc. and avoid treating it specially during evaluation.

Where does it come from?
https://imgur.com/a/oJMNy

Hi, it's me again, I found another thing which behaviours not the same as the c++'s version.

I think this is because the go join function doesn't behaviours the same as the c++ version's.
In the c++'s version, if the expression result is null, it will be ignored.
@sbarzowski
@sparkprime

Just asking because i am using this with kubernetes and trying to avoid any c++ dependencies.

Hi,

Thanks for open sourcing go-jsonnet library. Do you have any plan implementing the 'import' function in jsonnet language any time soon?

Thanks,
Jim

Current version completely ignores tailstrict

I found this in code:

case *valueFunction:
		return nil, makeRuntimeError("Couldn't manifest function in JSON output.", i.getCurrentStackTrace(trace))

and my input is something like this:

local a = import "../../../applib/a.libsonnet";
local b = import "../../../applib/b.libsonnet";

function(config={})
  local overall_config = default_config + config;
  (if a.match(overall_config) then
    {
      ...
    }
  else
    {
      ...
    }) + {
    ...
  }

And i get this:

RUNTIME ERROR: Couldn't manifest function in JSON output.
	During manifestation

How can i resolve this?
Thanks!

EvaluateSnippetMulti returns an "object" map[string]string, not an array.

Currently we don't detect missing arguments etc.

go vendor folder is missing.

Annotate individual end to end tests with things like:

• Whether it should succeed, end with runtime error or end with a static error.
• extvars and tlas
• error formatting settings (in case error is expected)
• native functions
• import callbacks
• Maximum stack
• Maximum number of printed frames on stack trace
• etc.

This is needed to properly test some features and it would be very helpful in unifying the Go and C++ test suites.

github.com/fatih/color is a small dep, but holds a larger dep tree. Coloring output is great for a terminal program, but not so great for a package.

I can work something up if you approve of the idea. I have a google CLA for the golang/go project already.