Git Product home page Git Product logo

starlark-go's Introduction

Starlark in Go

Go Tests Go Reference

This is the home of the Starlark in Go project. Starlark in Go is an interpreter for Starlark, implemented in Go. Starlark was formerly known as Skylark. The new import path for Go packages is "go.starlark.net/starlark".

Starlark is a dialect of Python intended for use as a configuration language. Like Python, it is an untyped dynamic language with high-level data types, first-class functions with lexical scope, and garbage collection. Unlike CPython, independent Starlark threads execute in parallel, so Starlark workloads scale well on parallel machines. Starlark is a small and simple language with a familiar and highly readable syntax. You can use it as an expressive notation for structured data, defining functions to eliminate repetition, or you can use it to add scripting capabilities to an existing application.

A Starlark interpreter is typically embedded within a larger application, and the application may define additional domain-specific functions and data types beyond those provided by the core language. For example, Starlark was originally developed for the Bazel build tool. Bazel uses Starlark as the notation both for its BUILD files (like Makefiles, these declare the executables, libraries, and tests in a directory) and for its macro language, through which Bazel is extended with custom logic to support new languages and compilers.

Documentation

Getting started

Build the code:

# check out the code and dependencies,
# and install interpreter in $GOPATH/bin
$ go install go.starlark.net/cmd/starlark@latest

Run the interpreter:

$ cat coins.star
coins = {
  'dime': 10,
  'nickel': 5,
  'penny': 1,
  'quarter': 25,
}
print('By name:\t' + ', '.join(sorted(coins.keys())))
print('By value:\t' + ', '.join(sorted(coins.keys(), key=coins.get)))

$ starlark coins.star
By name:	dime, nickel, penny, quarter
By value:	penny, nickel, dime, quarter

Interact with the read-eval-print loop (REPL):

$ starlark
>>> def fibonacci(n):
...    res = list(range(n))
...    for i in res[2:]:
...        res[i] = res[i-2] + res[i-1]
...    return res
...
>>> fibonacci(10)
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
>>>

When you have finished, type Ctrl-D to close the REPL's input stream.

Embed the interpreter in your Go program:

import "go.starlark.net/starlark"

// Execute Starlark program in a file.
thread := &starlark.Thread{Name: "my thread"}
globals, err := starlark.ExecFile(thread, "fibonacci.star", nil, nil)
if err != nil { ... }

// Retrieve a module global.
fibonacci := globals["fibonacci"]

// Call Starlark function from Go.
v, err := starlark.Call(thread, fibonacci, starlark.Tuple{starlark.MakeInt(10)}, nil)
if err != nil { ... }
fmt.Printf("fibonacci(10) = %v\n", v) // fibonacci(10) = [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]

See starlark/example_test.go for more examples.

Contributing

We welcome submissions but please let us know what you're working on if you want to change or add to the Starlark repository.

Before undertaking to write something new for the Starlark project, please file an issue or claim an existing issue. All significant changes to the language or to the interpreter's Go API must be discussed before they can be accepted. This gives all participants a chance to validate the design and to avoid duplication of effort.

Despite some differences, the Go implementation of Starlark strives to match the behavior of the Java implementation used by Bazel and maintained by the Bazel team. For that reason, proposals to change the language itself should generally be directed to the Starlark site, not to the maintainers of this project. Only once there is consensus that a language change is desirable may its Go implementation proceed.

We use GitHub pull requests for contributions.

Please complete Google's contributor license agreement (CLA) before sending your first change to the project. If you are the copyright holder, you will need to agree to the individual contributor license agreement, which can be completed online. If your organization is the copyright holder, the organization will need to agree to the corporate contributor license agreement. If the copyright holder for your contribution has already completed the agreement in connection with another Google open source project, it does not need to be completed again.

Stability

We reserve the right to make breaking language and API changes at this stage in the project, although we will endeavor to keep them to a minimum. Once the Bazel team has finalized the version 1 language specification, we will be more rigorous with interface stability.

We aim to support the most recent four (go1.x) releases of the Go toolchain. For example, if the latest release is go1.20, we support it along with go1.19, go1.18, and go1.17, but not go1.16.

Credits

Starlark was designed and implemented in Java by Jon Brandvein, Alan Donovan, Laurent Le Brun, Dmitry Lomov, Vladimir Moskva, François-René Rideau, Gergely Svigruha, and Florian Weikert, standing on the shoulders of the Python community. The Go implementation was written by Alan Donovan and Jay Conrod; its scanner was derived from one written by Russ Cox.

Legal

Starlark in Go is Copyright (c) 2018 The Bazel Authors. All rights reserved.

It is provided under a 3-clause BSD license: LICENSE.

Starlark in Go is not an official Google product.

starlark-go's People

Contributors

aarzilli avatar adonovan avatar aleksi avatar alexandear avatar algebra8 avatar b5 avatar cuishuang avatar d6o avatar dmitshur avatar emcfarlane avatar fenollp avatar jayconrod avatar josharian avatar kostya-sh avatar laurentlb avatar marco6 avatar mbrukman avatar mohamedelqdusy avatar nicks avatar pims avatar quarz0 avatar samwheating avatar tetromino avatar tetsuok avatar vladmos avatar vtsao avatar wojciechp avatar wyverald avatar yuikns avatar yuxincs avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

starlark-go's Issues

typo in syntax/scan.go

RBRACE should print as "}". Currently it prints "]", the RBRACK token.

--- a/syntax/scan.go
+++ b/syntax/scan.go
@@ -147,7 +147,7 @@ var tokenNames = [...]string{
        LBRACK:        "[",
        RBRACK:        "]",
        LBRACE:        "{",
-       RBRACE:        "]",
+       RBRACE:        "}",
        LT:            "<",
        GT:            ">",
        GE:            ">=",

repr(1.0) should be "1.0", not "1"

Python chooses to always add a decimal point when using str/repr on floats:

$ python2 -c 'print(repr(1.0))'
1.0
(exit 0)

$ python3 -c 'print(repr(1.0))'
1.0
(exit 0)

$ starlark-go -float -c 'print(repr(1.0))'
1
(exit 0)

In case you're curious, go-fuzz found this because Python and starlark disagree about whether int(repr(1.0)) is valid:

$ python2 -c 'int(repr(1.0))'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ValueError: invalid literal for int() with base 10: '1.0'
(exit 1)

$ python3 -c 'int(repr(1.0))'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ValueError: invalid literal for int() with base 10: '1.0'
(exit 1)

$ starlark-go -float -c 'int(repr(1.0))'

(exit 0)

Though there's nothing in the spec that says that the decimal point must be present for floats, it's probably preferable from a debugging perspective.

Status: needs decision

Add support for keyword-only arguments

Because unnamed arguments on BUILD rules are never a good idea for BUILD code health but are difficult for rule authors to disallow without jumping through hoops today.

A rule definition should be able to look like this:

def rule_name(*, name, parameter, optional_thing=None):
  if optional_thing is None:
    optional_thing = {}
  ...

Instead of the unnatural workaround that nobody seems to realize they need to write of:

def rule_name(**kwargs):
  name = kwargs.pop('name')
  parameter = kwargs.pop('parameter', [])
  optional_thing = kwargs.pop('optional_thing', {})
  if kwargs:
    # Manually fail with an error message about unknown arguments.
  ...

b/70686810 internally.

spec: allow parentheses on LHS of augmented assignment?

Consider:

a = 1
(a) += 2

Python 2 and 3 both accept this, yielding a == 3.

Starlark rejects it. The spec says of augmented assignments: "The left-hand side must be a simple target: a name, an index expression, or a dot expression."

The Go implementation panics when evaluating this code. I'd like to fix the panic, but first I'd like to double-check whether I should fix it by rejecting the code per existing spec or by accepting the code to match Python, with appropriate spec adjustments. (Accepting the code is a trivial implementation change; rejecting it without panicking probably also is.)

[feature request] while statement

As discussed here I'd like to have a while statement in the language, guarded by a global flag. The discussion in that issue suggests using the for statement but I don't think that's possible since the in operator makes it ambiguous.

If this is acceptable, I can try implementing this myself.

Nits found by various static check tools

$ unconvert ./...
internal/chunkedfile/chunkedfile.go:61:18: unnecessary conversion
starlark/int.go:43:42: unnecessary conversion
starlark/value.go:427:16: unnecessary conversion
$ ineffassign .
starlark/library.go:286:2: ineffectual assignment to ok
$ staticcheck  ./... |grep -v ST1006 # ignore different receiver names
internal/compile/compile.go:1:1: package comment should be of the form "Package compile ..." (ST1000)
repl/repl.go:1:1: package comment should be of the form "Package repl ..." (ST1000)
starlark/hashtable.go:284:22: func (*hashtable).dump is unused (U1000)
starlark/library.go:140:6: func builtinMethodOf is unused (U1000)
starlark/library.go:1809:6: func string_lstrip is unused (U1000)
starlark/library.go:1867:6: func string_rstrip is unused (U1000)
starlark/value.go:1100:7: identical expressions on the left and right side of the '!=' operator (SA4000)
starlark/value.go:1115:7: identical expressions on the left and right side of the '!=' operator (SA4000)
syntax/scan.go:380:2: field triple is unused (U1000)

I can file a PR to fix these. Just wondering if you had any opinions on them. For example, the identical expressions are using y != y directly instead of math.IsNaN(). Will string_lstrip will be used in the future or can it be removed? The hashtable.dump method is probably for debugging, so I'll leave that. Package comment tweaks are minor and inoffensive.

Build a Skylark playground

A playground similar to the Go Playground (play.golang.org) would help users experiment with Skylark, edit toy programs, and share and bookmark snippets of code. We should build one. The Skylark interpreter should be forked as a containerized subprocess of the web server so that it can be limited in its capacity to consume CPU and memory.

support kwonlyargs

Starlark in Java, like Python3 but not Python3, has the concept of keyword-only arguments (kwonlyargs).

This statement:
def f(a, b, *args, c=1, d=2, **kwargs): pass
declares a function with two kwonlyargs, c and d, as does this one:
def f(a, b, *, c=1, d=2, **kwargs): pass
except the allows variadic calls and the second does not.

The representation of function values will need to record the length of the suffix of the parameter tuple that are kwonlyargs, and the call operator must reject calls that would assign to any of them using a positional argument.

Support break outside of function body

In Tilt we are using Starlark as a general scripting language. The changes that were merged in f9eba72 have enabled our users to write a script, with logic, that exists at the top level.

I've encountered the desire for a break statement at the top level too, to turn "if else" that look like this:

if param:
  build_one_way()
else:
  build_another_way()

In to this

if param
  build_one_way()
  break

build_another_way()

I recognize that this isn't allowed in Python, but I'm curious what your thoughts are on supporting this in pursuit of making Starlark a general scripting language.

Thanks! Happy to help out with PRs if you think this is something you want to support.

reject +0 as field identifier in string.format

$ starlark -c "'{+0}'.format(9)"
$ python3 -c "'{+0}'.format(9)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
KeyError: '+0'
$ python2 -c "'{+0}'.format(9)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
KeyError: '+0'

The spec says: The *field name* may be either a decimal number or a keyword. Decimal number isn't defined, although python rejects +0. starlark rejects other weird possible decimal numbers:

$ starlark -c "'{0.0}'.format(9)"
Traceback (most recent call last):
  cmdline:1: in <toplevel>
Error: attribute syntax x.y is not supported in replacement fields: 0.0
$ starlark -c "'{-0}'.format(9)"
Traceback (most recent call last):
  cmdline:1: in <toplevel>
Error: keyword -0 not found

So I think we should probably reject +0 (and +1 etc).

use strings.Builder

Once Go 1.12 has been released, strings.Builder will be available in all supported released Go versions. We can then start using it.

Question: documentation on parallel execution of Starlark scripts?

I understand Starlark is able to take advantage of Go's underylying concurrency power. I was wondering if there is an example of how that would be leveraged via Starlark?

Within the context of a single Starlark script it would seem there is no access to anything thread-like or go (keyword) within the code itself. But I can see that when a script is evaluated there is an opportunity to pass in a thread-local context.

Does this mean that if you want to leverage parallelism it's simply a matter of bootstrapping a Starlark program with a thread per eval of a script?

Thanks in advance for what is shaping up to be a powerful and impressive package.

a math library

Starlark seem mighty terrific. I just have a minor query about doing some math.

In starlark-go, how would one raise n to the m-th power?

In Go, I would call math.Pow(n, m). In Python, math.pow(n, m).

Another way of asking: is there a way to get at the math library from Go perhaps?

strings have 'elems' attribute

Mismatch vs python. I haven't dug yet to understand whether this is benign.

$ starlark -c "getattr('a', 'elems')"
15:28:40 ~/src/go.starlark.net/fuzz/corpus (fuzz)$ python2 -c "getattr('a', 'elems')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: 'str' object has no attribute 'elems'
15:28:42 ~/src/go.starlark.net/fuzz/corpus (fuzz)$ python3 -c "getattr('a', 'elems')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: 'str' object has no attribute 'elems'

Found by go-fuzz.

should not accept "~=5"

$ starlark -bitwise -c '~=5'
$ starlark -bitwise -c 'print(~=5)'
-6
$ python2 -c 'print(~=5)'
  File "<string>", line 1
    print(~=5)
           ^
SyntaxError: invalid syntax
$ python3 -c 'print(~=5)'
  File "<string>", line 1
    print(~=5)
           ^
SyntaxError: invalid syntax

Maybe there's a correct way to parse ~=5 that I'm not seeing...but I don't think so. Alan?

Issue tracker link in README is 404.

The README currently says:

But that link is 404.

See:

https://github.com/google/starlark-go/blame/8474648b1aabb3712d0f2d2100a46c9974600188/README.md#L44

I'm not sure if the URL was changed from https://github.com/google/starlark/issues to https://go.starlark.net/starlark/issues unintentionally as part of a global replace in 6beab7e, or if that URL is meant to exist and redirect to GitHub. Hence an issue rather than PR.

starlark: reject duplicate keyword arguments in function call even when callee has **kwargs

$ cat a.star
def f(*args, **kwargs):
        print(args, kwargs)

f(x=1, y=2)
f(x=1, x=2)

$ starlark a.star
() {"x": 1, "y": 2}
() {"x": 2} # should be an error

Python2, Python3 and Starlark-in-Java all reject repetition of a keyword argument in a function call. Starlark-in-Go should to. (It does when the callee is a regular function but not when it has a **kwargs parameter.)

Time and Duration support?

I know bazel's skylark eschews time functions because they want to ensure reproducibility in their configurations, but this implementation of skylark seems like it could be used for so much more, and at least having access to official skylark time and duration types would keep implementations compatible that otherwise would not be, if consumers all create their own implementations.

Even purely from a configuration standpoint, I could definitely see wanting to be able to specify durations for timeouts, or timestamps for expiration dates, etc.

loose correctness check for unicode code points in strings

$ python2 -c '"\x-0"'
ValueError: invalid \x escape
$ python3 -c '"\x-0"'
  File "<string>", line 1
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 0-1: truncated \xXX escape
$ starlark -c '"\x-0"'

Haven't looked into why, but I'm guessing we use strconv on -0 and it happily returns 0.

Maybe I'm missing something, but I couldn't find the relevant part of the spec.

Possible bug: inc of a global variable that should be frozen on load

I'm not familiar with all the immutability rules of Starlark yet...but perhaps this a bug??

mod.star

x = 5
y = 10

app.star

load("mod.star",  "x",  "y")

# Errors out due to being frozen (as expected)
x = x + 1

# Does not error out, and succeeds
y+=1

print(x) # 5 since program errors out and therefore immutability is preserved.
print(y) # 11 since immutability was not preserved.

should not be able to assign to predeclared identifiers

$ python2 -c 'None=0'
  File "<string>", line 1
SyntaxError: cannot assign to None
(exit 1)

$ python3 -c 'None=0'
  File "<string>", line 1
SyntaxError: can't assign to keyword
(exit 1)

$ starlark-go -c 'None=0'

(exit 0)

starlark should reject None=0.

The spec agrees (see final sentence):

At the root of the tree is the predeclared block, which binds several names implicitly. The set of predeclared names includes the universal constant values None, True, and False, and various built-in functions such as len and list; these functions are immutable and stateless. An application may pre-declare additional names to provide domain-specific functions to that file, for example. These additional functions may have side effects on the application. Starlark programs cannot change the set of predeclared bindings or assign new values to them.

internal error with trailing tabs

The following script:

def f(n):
	if True:
		print("hello world")
		

Causes this error:

in.star:4:3: internal error: runtime error: index out of range

note that the last line contains two tab characters.

Getting a callstack from a bunch of nested load(...)'s

Imagine a bunch of files:

a.star:

...
load("b.star", "b")

b.star:

...
load("c.star", "c")
...

c.star:

...
def f():
  None()  # or some other runtime error
f()
...

What I'd like to see via Backtrace() when ExecFile("a.star", ...) is something like:

Traceback (most recent call last):
    a.star:1: in <toplevel>
    b.star:1: in <toplevel>
    c.star:4: in <toplevel>
    c.star:3: in f
  Error: invalid call of non-function (NoneType)

What I see is

Traceback (most recent call last):
    a.star:1: in <toplevel>
  Error: cannot load b.star: cannot load c.star: invalid call of non-function (NoneType)

Which is less helpful, since there's no pointer to where exactly the failure happened.

This appears to be due to stringification of errors from Load implementation here.

If my Load implementation reuses *starlark.Thread (and thus inherits its call stack), and If I change compile.LOAD to sniff out and pass through *EvalError unwrapped, then I get the backtrace I want, but I'm not sure it is a proper fix, considering how results of Load (including errors) in general are supposed to be cached, and the full error backtrace may be irrelevant if the error is picked up from the cache.

Do you have any suggestions?

reject dict(None)

$ python2 -c 'print(dict(None))'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: 'NoneType' object is not iterable
(exit 1)

$ python3 -c 'print(dict(None))'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: 'NoneType' object is not iterable
(exit 1)

$ starlark -c 'print(dict(None))'
{}
(exit 0)

The spec says:

dict creates a dictionary. It accepts up to one positional argument, which is interpreted as an iterable of two-element sequences (pairs), each specifying a key/value pair in the resulting dictionary.

None is not iterable, so dict should reject it.

To confirm:

$ starlark -c "reversed(None)"
Traceback (most recent call last):
  cmdline:1: in <toplevel>
Error: reversed: for parameter 1: got NoneType, want iterable

Fuzzing starlark

Spawned from #29 (comment)

I've been using the ExecFile example as by base:

// +build gofuzz

package starlark

import "go.starlark.net/resolve"

func Fuzz(data []byte) int {

	resolve.AllowFloat = true
	resolve.AllowSet = true
	resolve.AllowLambda = true
	resolve.AllowNestedDef = true
	resolve.AllowBitwise = true
	resolve.AllowGlobalReassign = true

	thread := &Thread{
		Print: func(_ *Thread, msg string) {},
	}

	predeclared := StringDict{}

	_, err := ExecFile(thread, "apparent/filename.star", string(data), predeclared)
	if err != nil {
		return 0
	}

	return 1
}

first with all the resolve flags on, then with them all off.

I used testdata/*.star as by seed corpus, which now has 2824 entries. I will file individual issues for the crashers.

reject using unhashable types with 'in'

$ starlark -c "{} in {}"
$ python3 -c "{} in {}"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: unhashable type: 'dict'
$ python2 -c "{} in {}"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: unhashable type: 'dict'
$ starlark -c "print({} in {})"
False

This might not be reasonably fixable. This comment from eval.go:777 is apropos:

// Ignore error from Get as we cannot distinguish true
// errors (value cycle, type error) from "key not found".

We could detect unhashable types by calling Hash before calling Get, although that would be expensive.

We could have a designated Unhashable error type, which we could ask implementations of Hash to return.

Or we could call this unfortunate and give up. Alan, preferences?

should not accept "int('0Oxa',8)"

$ starlark -c "int('0Oxa',8)"
$ python2 -c "int('0Oxa',8)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ValueError: invalid literal for int() with base 8: '0Oxa'
$ python3 -c "int('0Oxa',8)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ValueError: invalid literal for int() with base 8: '0Oxa'
$ starlark -c "print(int('0Oxa',8))"
10

Again, maybe this is right in some subtle way I'm not seeing. Alan?

should not accept parenthesized stmts

$ starlark -c "(print())"

$ python -c "(print())"
  File "<string>", line 1
    (print())
         ^
SyntaxError: invalid syntax

I'm not sure exactly where the error is here, but something is not right. :)

Found by go-fuzz, of course.

Add spell check to "value has no .f field" errors

It would help the user if when a field name x.foo is mistyped, the error message gives a hint as to the correct name (did you mean food?). Since we're already producing an error, performance is not really an issue. The challenge is to do it without adding a dependency on a Levenshtein or Soundex package.

repeated slicing of ranges mishandled

$ starlark -c "print(range(3)[::2][:0][0])"
0
$ python2  -c "print(range(3)[::2][:0][0])"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
IndexError: list index out of range
$ python3  -c "print(range(3)[::2][:0][0])"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
IndexError: range object index out of range

This is an interesting bug. It appears (from the behavior) that it has to do with some hidden state in the range object or a compiler error:

$ starlark
Welcome to Starlark (go.starlark.net)
>>> range(3)
range(3)
>>> range(3)[::2]
range(0, 3, 2)
>>> range(3)[::2][:0]
range(0, 0, 2)
>>> range(3)[::2][:0][0]
0
>>> range(0, 0, 2)[0]
Traceback (most recent call last):
  <stdin>:1: in <expr>
Error: range index 0 out of range [0:0]

Note the difference in behavior between constructing range(0, 0, 2) via range(3)[::2][:0] vs spelling it directly as range(0, 0, 2).

The same python3 session, for reference:

$ python3
Python 3.7.2 (default, Jan  8 2019, 11:26:15) 
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> range(3)
range(0, 3)
>>> range(3)[::2]
range(0, 3, 2)
>>> range(3)[::2][:0]
range(0, 0, 2)
>>> range(3)[::2][:0][0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: range object index out of range
>>> range(0, 0, 2)[0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: range object index out of range

UnpackArgs: how to detect explicitly set parameters

# foo.star

fn(v = 0)
fn()
// bar.go

var v int
starlark.UnpackArgs(r.Name(), args, kwargs, "v?", &v)
// How can we test if `v` has been explicitly set or has its default value?

Analogy with go encoding/json strategy:

var v *int
json.Unmarshal([]byte(""), &v) 
// or
json.Unmarshal([]byte("0"), &v
// then
if v == nil {...}

doc: show examples of UnpackArgs

Example: all arguments must be named:

if len(args) > 0 {
    return nil, fmt.Errorf("%s: unexpected positional arguments")
}
if err := UnpackArgs(b, args, kwargs, ...); err != nil {
    return nil, err
}

Example: equivalent to f(x, y=1, *, z=2):

if len(args) > 2 {
    return nil, fmt.Errorf("%s: unexpected positional arguments")
}
var x, y, z int = 0, 1, 2
if err := UnpackArgs(b, args, kwargs, "x", &x, "y?", &y, "z", &z); err != nil {
    return nil, err
}

bug in dict.star tests

https://github.com/google/starlark-go/blob/master/starlark/testdata/dict.star#L76

I think the blocks of x10 and x11 should be updated to use those newly created values?

# dict.get
x10 = {"a": 1}
assert.eq(x9.get("a"), 1)
assert.eq(x9.get("b"), None)
assert.eq(x9.get("a", 2), 1)
assert.eq(x9.get("b", 2), 2)

# dict.clear
x11 = {"a": 1}
assert.contains(x10, "a")
assert.eq(x10["a"], 1)
x10.clear()
assert.fails(lambda: x10["a"], 'key "a" not in dict')
assert.true("a" not in x10)
freeze(x10)
assert.fails(x10.clear, "cannot clear frozen hash table")

repl rejects multiline strings

Sadly it is difficult to step through code line-by-line at the repl when the code contains multiline strings such as docstrings.

$ starlark
Welcome to Starlark (go.starlark.net)
>>> a = """trying to enter a multiline string at the repl 
... that has two newlines after this line...
...
<stdin>:1:5: unexpected EOF in string
>>> # arg.

skylark: don't access runtime.stringHash

Russ Cox points out that CL google/skylark@f94b021 makes Skylark depend on the internals of one particular Go implementation. In its defense, stringHash is one of the most stable runtime functions, its disappearance would be immediately detected by build/test using a new Go release, its assembly implementation is not a trivial piece of code, and a pure-software workaround is readily available. That all said, it does cut off use of skylark from GopherJS, for example, and possibly also gccgo.

The task of this issue is to make this dependency cleaner, either by duplicating the AESENC assembly from the Go runtime, or by build-tagging the non-portable code so that the runtime internals are only used when known to be available.

See also golang/go#28322

[feature request] Support python-like docstrings

The language already allows an statement consisting only of a literal string, to appear as the first statement of a function. I propose to add a Doc() method to *Function that will return the value of this string.

Backwards compatibility: this doesn't change the syntax of starlark nor its behavior.

Motivation: I want to use starlark as scripting language for delve, I intend to scan the list of global variables returned by ExecFile and bind any global function that starts with "command_" to a command accessible from delve's command line. All the predefined commands have a documentation string that is consumed by the "help" command. I would like starlark-defined commands to also have this.

Alternatives: collect the comments preceding the function definition and return them somehow. Looking at the syntax package it seems to do some of this already but I don't think it's exposed anywhere, also poking at it with a debugger it doesn't look like the commentsRef of DefStmt actually ever gets set to anything (but maybe I just don't get what it's supposed to do).

Mismatch with Python: reversed(x) is subscriptable

Both Python2 and Python3 reject reversed([1, 2])[0] and reversed([1, 2])[0] = 6. starlark accepts both. This is an immediate consequence of the spec:

reversed(x) returns a new list [...]

I don't really think we want to change the starlark behavior, but since it does disagree with both Pythons, I did want to call it out for an explicit decision.

should reject "print(S=5)"

$ starlark -c "print(S=5)"
S=5
$ python -c "print(S=5)"
  File "<string>", line 1
    print(S=5)
           ^
SyntaxError: invalid syntax

Possibly a duplicate of #53. Found by go-fuzz.

unclear error on mismatched parens

Starlark:

$ starlark -c "a = max(range(10))); print(a)"
cmdline:1:19: indentation error

Python:

$ python -c "a = max(range(10))); print(a)"
  File "<string>", line 1
    a = max(range(10))); print(a)
                      ^
SyntaxError: invalid syntax

"invalid syntax" is a lot better than "indentation error". (Even better would be "mismatched parens".)

index (and rindex) do not advance sufficiently when calling with ""

$ python2 -c "print('a'.index('', 50))"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ValueError: substring not found
(exit 1)

$ python3 -c "print('a'.index('', 50))"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ValueError: substring not found
(exit 1)

$ starlark-go -c "print('a'.index('', 50))"
1
(exit 0)

Definitely a corner case: only occurs when searching for the empty string and the occurrence requested is higher than the length of the input string.

wip-stargo: varargs

I'm still seeing oddness with varargs at c9f53cb

commit c9f53cb (HEAD -> wip-stargo, origin/wip-stargo, stargo-wip)
Author: Alan Donovan [email protected]
Date: Sat Jan 5 17:37:51 2019 -0500

staranise: prototype of dynamically loaded vet checks

Change-Id: I88be67d55ffa6b33e83dff29968f8cdacfda3f64

>>> load("go", fmt="fmt")
>>> fmt.Printf("hi\n")
Traceback (most recent call last):
  <stdin>:1: in <expr>
Error: in call to fmt.Printf, got 1 arguments, want 2
>>>

Alan commented on the list:

Thanks, fixed now. (I'm continually rewriting the history of the wip-stargo branch, BTW).

I think c9f53cb is the latest. But with the history rewrite I can't really tell for sure. Git deletes the actual dates.

inconsistent acceptance of trailing commas in argument lists

$ starlark
Welcome to Starlark (go.starlark.net)
>>> def f(*args, **kwargs): pass
... 
>>> f(1,)
>>> f(*(1,2),)
... 
<stdin>:1:11: got ')', want argument
>>> f(a=1,)
>>> f(**{"a": 1},)
... 
<stdin>:1:15: got ')', want argument
>>>  

Observe that for plain positional and keyword args, a trailing comma is accepted. The other two cases are not.

Python 2 behaves the same as starlark-go; Python 3 accepts all forms.

I prefer the Python 3 behavior.

$ python2
Python 2.7.15 (default, Jun 17 2018, 12:46:58) 
>>> def f(*args, **kwargs): pass
... 
>>> f(1,)
>>> f(*(1,2),)
  File "<stdin>", line 1
    f(*(1,2),)
             ^
SyntaxError: invalid syntax
>>> f(a=1,)
>>> f(**{"a": 1},)
  File "<stdin>", line 1
    f(**{"a": 1},)
                ^
SyntaxError: invalid syntax
$ python3
Python 3.7.1 (default, Nov 14 2018, 12:09:13) 
>>> def f(*args, **kwargs): pass
... 
>>> f(1,)
>>> f(*(1,2),)
>>> f(a=1,)
>>> f(**{"a": 1},)

I haven't checked yet what the spec mandates.

missing check that all arguments consumed during string formatting

$ starlark -c "(''%0)"
$ python2 -c "(''%0)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: not all arguments converted during string formatting
$ python3 -c "(''%0)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: not all arguments converted during string formatting

Found by go-fuzz.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.