cue-lang / cue Goto Github PK

Originally raised by @takirala in cuelang/cue#48

Originally raised by @sttts in cuelang/cue#43

Originally opened by @sah4ez in cuelang/cue#32

I have cue file test.cue:

address: string

And when I call this, I got an error:

$ cue export test.cue 
json: error calling MarshalJSON for type cue.Value: json: error calling MarshalJSON for type cue.Value: cannot convert incomplete value "string" to JSON

Originally opened by @enisoc in cuelang/cue#20

Since CUE is a superset of JSON, I understand that you can paste valid JSON into a .cue file and it will be guaranteed valid CUE as well.

My question is, is it guaranteed that valid JSON pasted into a .cue file will never accidentally invoke/trigger any features of CUE that are not in JSON?

For example, I'm guessing that string interpolation can't be triggered accidentally by valid JSON, since (I presume) the escape sequence \( is invalid JSON. Is CUE designed in such a way that all such features are systematically impossible to trigger from valid JSON?

In other words, is all valid JSON guaranteed to produce exactly the same output when treated as a CUE program? I haven't seen mention of this in the docs.

Originally opened by @mpvl in cuelang/cue#14

kubectl allows patching using a "strategic merge". In short, it allows lists to be treated like maps so that the right elements can be merged.

In general, unification of lists is very tedious for automated configuration. A possible solution is for users to define mappings manually. Although CUE can handle reconstructing and merging unknown API, it is still tedious. Ideally users would just use the native API of whatever system they work on.

Also, ideally, mappings should be able to be applied with the same separation of concerns as is possible with normal objects.

One possible approach would be to extend the emit mechanisms for inner objects:

myList: [ ...{ name: string } ]

// before evaluation, the list is converted to a map like this
myList: {
    <- "\(x.name)" : x for x in $ // $ means self or current object
    // after evaluation, it is converted back to this format.
    -> [ x for x in $ ]
}

The top-level emit is then -> Expr where the -> can be elided as the top-level value is always a map.

This needs a lot more thought, but having some kind of mechanism like this can be important.

There may also be overlap with a design for field attributes.

Originally opened by @garethr in cuelang/cue#39

The current output points folks to http://cuelang.org/pkg/tool for information on available tasks. However that URL doesn't have any content.

$ curl -i http://cuelang.org/pkg/tool
HTTP/1.1 404 Not Found
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
X-Cloud-Trace-Context: 06cb78cc27dfe93e8cc9dd3bde03b522;o=1
Date: Mon, 22 Apr 2019 10:32:57 GMT
Server: Google Frontend
Content-Length: 19

404 page not found

Originally opened by @mpvl in cuelang/cue#29

Referential cycles are currently implemented. Structural cycles are not, however. So CUE currently allows the definition of infinite structures like:

{ f: { a: f } }

As defined by the spec, infinite structures are not allowed. Infinite structures should be detected and disallowed using an occurs check, as is commonly done in graph unification implementations.

There are some design decisions to be made though. A careful choice of semantics in the occurs check could make CUE not Turing complete and even guarantee termination. This weakens the power of CUE, but may be a desirable property if CUE is to be used for things like smart contracts, where the ability to determine termination would be a major win.

For an example of Turing completeness in CUE see the test called "reentrance" in resolve_test.go, which implements Fibonacci in CUE. The specifics of the semantics of the occurs check will determine whether these shenanigans would be allowed or not.

Depending on the level of restrictions, it should be investigated whether the occurs check can be implemented cheaply using Tomabechi-style cycle detection.

Originally opened by @mpvl in cuelang/cue#26

Analogous to the Go's go generate.

CUE's is designed with code generation in mind and it is to be expected that many CUE files will be generated. A cue:generate directive would allow annotating the methods used to generate the files in a way familiar to Go programmers.

Originally opened by @fabxc in cuelang/cue#41

I can't seem to get import of a local package to work. An example would be great. Maybe that could be folded into kubernetes/manual?

Originally opened by @wxio in cuelang/cue#27

Please take a look at https://github.com/jpillora/opts/tree/subcommands (work in progress). I think cue is a fascinating project and happy to contribute (I have started a similar project using protobufs as starting point).

Thought helping move from Cobra to Opts might be a good first start.

Related to #25, #9

One of this nice things about the work in progress on Opts is basic bash, zsh and fish command completion is free and highly customisable for more complex requirements.

Originally opened by @enisoc in cuelang/cue#21

As of current master, the following CUE program doesn't evaluate as expected:

import "encoding/json"

input: "{\"one\": 1, \"two\": 2, \"three\": 3}"
result: json.Unmarshal(input)

Expected output:

{
    "input": "{\"one\": 1, \"two\": 2, \"three\": 3}",
    "result": {
        "one": 1,
        "two": 2,
        "three": 3
    }
}

Actual output (cue export):

{
    "input": "{\"one\": 1, \"two\": 2, \"three\": 3}",
    "result": {
        "Valid": 1,
        "Compact": 2,
        "Indent": 3
    }
}

Notice the field names in the unmarshaled result have been replaced with the names of methods in pkg/encoding/json.

Originally raised by @gkleiman in cuelang/cue#47

Originally opened by @wstrange in cuelang/cue#38

I am not sure if this is pilot error, or a recent commit changed something, but I can't see any objects being listed or dumped.

cd cuelang/cue/doc/tutorial/kubernetes/manual/services
cue cmd dump ./...
cue cmd dump
cue cmd ls 
cue cmd ls ./...

None of the above commands produce any output. cmd ls|dump works if I specify a path to a directory. For example - this works:

cue ls frontend/waiter 
Service         frontend        waiter
Deployment      frontend        waiter

Is this expected behavior?

Originally opened by @mpvl in cuelang/cue#8

Allow diffing between (snapshots of) previous version and current version.

Originally opened by @mpvl in cuelang/cue#10

• keeping the links in sync is kind of tedious.
• would be nice to have some proper formatting
• use in-browser evaluation of example (using the GopherJS port)

Originally opened by @trebe in cuelang/cue#31

Thanks for the nice walkthrough, that shows some of the great power of Cue :-)
Parts of it refers to previous versions of Cue:

cue eval ./... > snapshot2 doesn't fail as indicated, since now eval can output non-atomic string

Under "1 is the default", 1 is not marked as default.

Originally opened by @garethr in cuelang/cue#30

I was experimenting with adding a set of validation commands to my cue configuration, in _tools.cue files. The commands they exec out to will return non-zero exit codes when those commands fail, but running them via the cue <cmd> runner eats those exit codes and always returns 0. Checking the code this doesn't appear to be something I can add to the print task.

https://github.com/cuelang/cue/blob/2b0e7cd9f63a190e762d7c802b98528ff80dcb7c/cmd/cue/cmd/custom.go#L299-L310

Is there a way of achieving this at present? Supporting passing through the exit codes would make integration of cue commands into pipelines easier.

Originally opened by @mpvl in cuelang/cue#9

This likely means that the cli docs should be generated from the tool code.

Originally opened by @mpvl in cuelang/cue#24

This is a tracking issue related to extracting CUE from Go code. The functionality in this issue focusses on use cases where Go is used as the source of truth. Issue #6 focusses on generating Go assuming that CUE is used as the source of truth.

injection:
Allow annotation of Go types with additional CUE constraints and provide a simple API to validate and complete Go values.

extraction:
The primary extraction method looks at struct definitions in Go code, determines how these would be interpreted by the encoding/json package, and then generates CUE definitions for such structs.
In addition, we can extract CUE annotations used in Go code for validation to augment such definitions.

Originally opened by @trebe in cuelang/cue#23

With this code

x: {a:1}|{a:2}
y: x & {a:3}

I get the following not so helpful message:

empty disjunction: &{{0xc0001fe600} <nil> <nil> [] [{4 0xc00007ee00 <nil> <nil>}] 0xc0001eed20}:
    /...<filename>:2:4
    
terminating because of errors

Originally opened by @mpvl in cuelang/cue#12

Allow constraining string values with regular expressions:

a: re("[a-z][a-z][a-z]")
a: "foo"

The syntax is yet to be determined. But see #1.

Originally opened by @mpvl in cuelang/cue#5

Tracking issue for adding protobuf support to CUE:

• import command reads text and binary proto instance
• import command reads proto definitions
• export command supports the converse
• core libs

Originally raised by @alenkacz in cuelang/cue#46

Originally opened by @mpvl in cuelang/cue#15

When converting an overlay system to use CUE, "parent" template will often hard code defaults, about which CUE will complain if any instances unify a struct overriding such defaults. As this is likely a common case, it makes sense to add refactor support for this.

The relax command can rewrite parent structs to relax constraints. It is a simple variant of anti-unification as proposed in #7.

In a typical workflow, a user would run relax and then manually verify in the vc diffs which relaxations to keep and which reveal bugs.

Originally opened by @patflynn in cuelang/cue#22

I was a little surprised to see the '.' here:
https://cue.googlesource.com/cue/+/HEAD/doc/tutorial/basics/rangedef.md

int16 >=-32_768 & <=.32_767

Originally opened by @enisoc in cuelang/cue#19

Not sure if this is just not implemented yet, but the examples of breaking reference cycles don't work as of current master:

// CUE knows how to resolve the following:
x: 200
x: y + 100
y: x - 100

constraint violated: _|_(200:delayed constraint (200 & (<0>.y + 100)) violated)

Originally opened by @mpvl in cuelang/cue#11

For instance:

a: >5         // This field must be greater than 5
b: !=null     // This field must not be null

c: >3.0 & <10  // (3.0, 10.0) open ended range on both sides

Supported operators:

< > >= <= != ==

These would be defined on the same types as their binary equivalents. Semantically such expressions would define disjunctions of possible values, analogous to ranges.

Introducing these would solve various limitations that are currently present. Such as defining open ended ranges for floats or requiring that a value should not be of a specific type.

Originally opened by @enisoc in cuelang/cue#17

The tutorial page on Bottom makes it seem like the user can see partial output like this when errors occur:

a:    _|_
l:    _|_
list: [0, 1, 2]
val:  _|_

But instead, the tool outputs messages like this:

cannot unify numbers 2 and 3:
    [...]
cannot unify numbers 4 and 5:
    [...]
index 3 out of bounds:
    [...]

From #13 it sounds like the tool just doesn't support this yet. In the meantime, perhaps the docs should be edited to clarify that the output shown is theoretical, so users know they didn't do something wrong.

Originally opened by @fabxc in cuelang/cue#42

cue seems to detect what the spec describes as reference cycles. But for structural cycles, it runs into a stack overflow:

goroutine 1 [running]:
runtime.heapBitsSetType(0xc04ab64070, 0x8, 0x8, 0xc5bcc0)
        /usr/lib/google-golang/src/runtime/mbitmap.go:938 +0xa55 fp=0xc028000350 sp=0xc028000348 pc=0x52cea5
runtime.mallocgc(0x8, 0xc5bcc0, 0x1, 0x0)
        /usr/lib/google-golang/src/runtime/malloc.go:969 +0x50e fp=0xc0280003f0 sp=0xc028000350 pc=0x522c3e
runtime.newobject(0xc5bcc0, 0x0)
        /usr/lib/google-golang/src/runtime/malloc.go:1068 +0x38 fp=0xc028000420 sp=0xc0280003f0 pc=0x523298
cuelang.org/go/cue.(*structLit).expandFields(0xc00025c000, 0xc01907de60, 0xc01907de60)
        /usr/local/google/home/freinartz/repos/src/cuelang.org/go/cue/value.go:646 +0x42 fp=0xc028000658 sp=0xc028000420 pc=0x7d88c2
cuelang.org/go/cue.Value.structValOpts(0xc000240b90, 0xc04ab63a00, 0xc01907de60, 0x101000000, 0xc04ab63a00, 0xc00025c000, 0xd98c20, 0xc00025c000, 0xd96780)
        /usr/local/google/home/freinartz/repos/src/cuelang.org/go/cue/types.go:813 +0xcd fp=0xc028000788 sp=0xc028000658 pc=0x7d239d
cuelang.org/go/cue.Value.structVal(...)
        /usr/local/google/home/freinartz/repos/src/cuelang.org/go/cue/types.go:799
cuelang.org/go/cue.Value.Walk(0xc000240b90, 0xc04ab63a00, 0xc047fff870, 0x0)
        /usr/local/google/home/freinartz/repos/src/cuelang.org/go/cue/types.go:1126 +0xe9 fp=0xc028000890 sp=0xc028000788 pc=0x7d4649
cuelang.org/go/cue.Value.Walk(0xc000240b90, 0xc04ab639c0, 0xc047fff870, 0x0)
        /usr/local/google/home/freinartz/repos/src/cuelang.org/go/cue/types.go:1129 +0x161 fp=0xc028000998 sp=0xc028000890 pc=0x7d46c1
cuelang.org/go/cue.Value.Walk(0xc000240b90, 0xc04ab63980, 0xc047fff870, 0x0)
        /usr/local/google/home/freinartz/repos/src/cuelang.org/go/cue/types.go:1129 +0x161 fp=0xc028000aa0 sp=0xc028000998 pc=0x7d46c1
cuelang.org/go/cue.Value.Walk(0xc000240b90, 0xc04ab63940, 0xc047fff870, 0x0)
...

Originally opened by @ghchinoy in cuelang/cue#49

https://github.com/cuelang/cue/blob/master/doc/tutorial/basics/stringraw.md

Note the markdown uses "```json" causing the red rendering

ie this...

$ cue eval stringraw.cue

{
    msg1: "The sequence \"\\U0001F604\" renders as 😄."
    msg2: """
        A regular expression can conveniently be written as:
        
            #\"\\d{3}\"#
        
        This construct works for bytes, strings and their
        multi-line variants.
        """
}

should be ...

$ cue eval stringraw.cue

{
    msg1: "the sequence \"\\U0001F604\" renders as 😄."
    msg2: """
        A regular expression can conveniently be written as:
        
            #\"\\d{3}\"#
        
        This construct works for bytes, strings and their
        multi-line variants.
        """
}

Similarly, https://github.com/cuelang/cue/blob/master/doc/tutorial/basics/bytes.md exhibits similar (missing quotes around cue export rendered json due to missing quotes in md)

ref also cuelang/cue#48

Originally raised by @hacfi in cuelang/cue#45

Originally opened by @mpvl in cuelang/cue#13

The output from eval is a mix of JSON and (invalid) CUE and indents weirdly. It was quickly written before package format was written.

Replace the output with using package format.

Also, it would be useful to see partially evaluated configurations.

Originally opened by @mpvl in cuelang/cue#28

CL https://cue-review.googlesource.com/c/cue/+/1660 introduced an alternative semantics for default values that allow also default values to be fully associative, commutative, idempotent, and distributive.

This still needs to be implemented. The implementation should go hand in hand with tidying up and clarifying the semantics of references and the use of defaults in arithmetic and the combination thereof.

Originally opened by @objectiveous in cuelang/cue#34

While running:
go get -u cuelang.org/go/cmd/cue

I get:
cmd/cue/cmd/vet.go:66:7: undefined: fVerbose

https://github.com/cuelang/cue/blob/master/cmd/cue/cmd/vet.go#L66

Originally opened by @mpvl in cuelang/cue#7

Implement cue hoist to extract templates from concrete data definitions. One could see this as cue trim in the opposite direction.

This can be implemented using the known anti-unification algorithm. It probably makes sense, though, to disregard disjunctions or to limit the number of elements.

Originally opened by @mpvl in cuelang/cue#2

There is currently no way to reference a top-level field with a non-standard character:

"foo-bar": "can't touch this"

a: // cannot reference "foo-bar"

One solution is to introduce $ to refer to the root object. This is possible, but such references often result in poor configuration file design. It is also one other thing to learn for a user.

An alternative approach is to allow a syntax for identifiers with non-standard characters. Swift and GCL allow back quotes for this purpose:

"foo-bar": "can touch this"

a: `foo-bar`

Main drawback: now we have three ways of referring to identifiers. Allowing back quotes also on the left hand side might mitigate this issue. The string approach may be considered a relic of JSON and hardly used unless needed for interpolation.

`foo-bar`: "can touch this"

a: `foo-bar`

"dynamic\(interpolated)": 42

cue fmt could be used to canonicalize LHS identifiers according to this rule.

Originally opened by @pblkt in cuelang/cue#40

The behavior of struct unification specified as

The successful unification of structs a and b is a new struct c which has all fields of both a and b

Cannot define an upper bound for structs. Specifically, CUE cannot help us verify that the resulting expression is limited to allowed fields.

For example, the unification

base: {
  onlyViableField: string | *"myDefault"
}

{
  base & {onlyViableFeild: "nonDefaultValue"}
}

yields

{
  onlyViableField: "myDefault"
  onlyViableFeild: "nonDefaultValue"
}

In kubernetes, the existence of extraneous fields causes failure when kubectl applys the resulting YAML, whereas in other systems the expression above could use silently use the default value erroneously.

When comparing CUE to a typed configuration language (like dhall), this is a qualitative difference in capabilities, since the extraneous field would be found by explicitly typing the result.

Extraneous fields can be hard to find - especially in large, opaque, auto-generated config schemes.

Originally raised by @metalmatze in cuelang/cue#44

Originally opened by @mpvl in cuelang/cue#4

Current CUE supports the more old-school 0777 notation. 0o, however, is more legible, has better properties regarding scanning, and seems to be the standard languages are converging on.

The CUE-YAML importer would need to be modified to convert octal numbers accordingly.

The old format should be dropped.

Originally opened by @mpvl in cuelang/cue#6

Or really any programming language, but Go is a good first target.

• Generate Go structs (with comments) from CUE definitions
• Generate validation code given the CUE constraints defined on these definitions (well, its the same thing in CUE, but for Go it needs to be split out)

Originally opened by @enisoc in cuelang/cue#18

I'm not sure if this is just not implemented yet, but filing an issue to track either way.

At current master, the examples for string interpolation in field names don't seem to work. Any field with an interpolation in the name is silently dropped from the output.

Originally opened by @mpvl in cuelang/cue#3

This is a tracking issue for the implementation of the commented out sections in the spec for conversions.

For instance:

• convert between the integer types
• between byte lists and bytes
• etc

But also to convert a struct to a specific struct type. The latter is quite handy for various purposes.

Originally opened by @ghchinoy in cuelang/cue#50

Using types.cue from https://github.com/cuelang/cue/blob/master/doc/tutorial/basics/types.md errors on running cue eval.

$ cue eval -i types.cue 
--- .
unsupported op &((float)*, int):
    ./types.cue:2:8
    ./types.cue:7:10
    
// /home/ghchinoy/dev/cueexperments
{
    point: {
        x:  float
        y:  float
    }
    xaxis: {
        x:  _|_ // unsupported op &((float)*, int)
        y:  float
    }
    yaxis: {
        x:  float
        y:  _|_ // unsupported op &((float)*, int)
    }
    origin: {
        x:  _|_ // unsupported op &((float)*, int)
        y:  _|_ // unsupported op &((float)*, int)
    }
}```

Originally raised by @garethr in cuelang/cue#37

Originally opened by @ngrilly in cuelang/cue#33

How is CUE different from jsonnet, both coming from Google, and the latter being largely promoted especially in the context of Kubernetes?

Originally opened by @sah4ez in cuelang/cue#35

https://github.com/cuelang/cue/blob/433ab7a9079af5410fa6d5270016fce533891799/cmd/cue/cmd/vet.go#L66
*fVerbose not exists...

Originally opened by @pavlos-christoforou in cuelang/cue#36

Excellent effort, would be a great replacement for our inhouse python based system. However we are missing an important feature, documenting fields which are accessible from api/programming which helps auto generating documentation, provides schema comments when autogenerating schemas for third party systems etc.

Originally opened by @mpvl in cuelang/cue#25

To display the current build and version.

Originally opened by @jba in cuelang/cue#16

a: [1] + [2]

gives

unsupported op +(list, list)

at current master (7d0797b).

Originally opened by @mpvl in cuelang/cue#1

Swift raw strings seem to be the nicest design for raw strings out there.

Benefits:

• just a generalization of normal strings, including multi-line
• allow string interpolation within raw strings
• frees up back quotes for, for instance, identifier literals (with non-standard characters).
• extends to both string and byte literals
• can still use escape sequences within the raw strings
• but typically does't require escaping any.

#"This is \ a raw string"#
#"Raw string with interpolation \#(interpolated)."#
##"Raw string without interpolation \#(interpolated)."##

etc.