ron-rs / ron Goto Github PK

Will RON ever support multi-line string literals? This is a feature that missing from JSON, but is present in YAML, TOML, and in Rust proper, like so:

# YAML multi-line string literals
key: >
  This is a very long sentence
  that spans several lines in the YAML
  but which will be rendered as a string
  without carriage returns.

# TOML multi-line string literals
key = '''
    This is a very long sentence
    that spans several lines in the TOML
    but which will be rendered as a string
    without carriage returns.
'''

// Rust multi-line string literals
let key = r#"
    This is a very long sentence
    that spans several lines in Rust
    but which will be rendered as a string
    without carriage returns.
"#;

let key = r##"
    This is a sentence which happens
    to contain a '#' character inside.
"##;

This could be a useful for storing localized strings for dialogue boxes, for example, or other complex strings that are too tedious and error-prone to escape by hand. Here is some pseudo-RON syntax showing how this feature could be used:

System(
    arch: "x86_64-linux",
    shell: "/usr/bin/bash",
    packages: ["firefox", "git", "rustup"],
    startup: r##"
        #!/usr/bin/bash
        export PS1="..."
        echo "Done booting!"
    "##,
)

Given that RON already mimics Rust's notation quite well, adopting its syntax for raw string literals would make this feature feel right at home. See the official reference page and Rust By Example for details.

Current view of a RON file in Github is rather unusable. Github uses Linguist for determining the syntax and applying the correct highlight. We should make an upstream feature to recognize RON.

Syntax: #![attribute_name] (like in Rust)

After reading https://ozkriff.github.io/2017-12-01--devlog.html where @ozkriff said that his configs (https://raw.githubusercontent.com/ozkriff/zemeroth_assets/master/objects.ron) aren't looking very nice because of how RON handles newtypes, I thought about adding attributes which allow to configure RON for your needs. Not only would this allow for toggles like #![unwrap_newtypes], but it could also be used to deal with new features. That way, we can just enforce that you need #![feature_name] to enable a RON feature, such that a parser without support for that feature can return a simple and understandable error.

A more concise format for arrays can really help:

Instead of this:

Scene( // class name is optional
    materials: { // this is a map
        "metal": (
            reflectivity: 1.0,
        ),
        "plastic": (
            reflectivity: 0.5,
        ),
    },
    entities: [ // this is an array
        (
            name: "hero",
            material: "metal",
        ),
        (
            name: "monster",
            material: "plastic",
        ),
    ],
)

You can do something more concise:

Scene( // class name is optional
    materials: { // this is a map
        "metal": (
            reflectivity: 1.0,
        ),
        "plastic": (
            reflectivity: 0.5,
        ),
    },
    entities: [ 
       {"name", "material", }, //Use a map/tuple to define what fields will be there in each item in the array
       ( "hero", "metal"),  //the fields are presented in the same order with repeating column names
       ( "monster", "plastic",),
    ],
)

The first item in the array is actually an optional map which can be used to specify the column names.

entities[0]["name"] will be "hero"
entities[0]["material"] will be "metal"

..and so on.

If the map defining the list of column names is not provided for an array, perhaps we can default to using integers for the column names indexed from 0.

Consider an example below without column names:

Scene( // class name is optional
    materials: { // this is a map
        "metal": (
            reflectivity: 1.0,
        ),
        "plastic": (
            reflectivity: 0.5,
        ),
    },
    entities: [       
       ( "hero", "metal"),  //the fields are presented in the same order with repeating column names
       ( "monster", "plastic",),
    ],
)

That is:
entities[0][0] will be "hero"
entities[0][1] will be "metal"
entities[1][0] will be "monster"
entities[1][1] will be plastic"

Using a map or a tuple in an array is definitely a bit ugly - perhaps an alternate syntax can be used but it does significantly reduce the overall size of the data especially if there are 100s (or more) items in an array.

It is also conceivable that the map describing the columns could be made much more sophisticated.

Example:

Instead of:

  
"entities": [{"name", "material"},
     ("hero", "metal", ),
     ("monster", "plastic",),
 ],

The column definitions could be made much more detailed.

 "entities": [
            { "name": {
                 "type": "char",
                 "length": 10,
               },
              "material": {
                 "type": "char",
                 "length": 30,
              },
            },
          ("hero", "metal"),
          ("monster", "plastic"), 
  ]

I'm pretty sure unicode support isn't fully implemented, so we would need tests and full support in serializer and deserializer. Additionally, rules should go into the spec (probably in a text document in this repository, as suggested somewhere).

test:

extern crate ron;
#[macro_use]
extern crate serde_derive;

#[derive(Serialize, Deserialize)]
pub struct ImVec2 {
    pub x: f32,
    pub y: f32,
}

#[derive(Serialize, Deserialize)]
pub struct ImColorsSave {
    pub text: f32,
}

#[derive(Serialize, Deserialize)]
pub struct ImGuiStyleSave {
    pub alpha: f32,
    pub window_padding: ImVec2,
    pub window_min_size: ImVec2,
    pub window_rounding: f32,
    pub window_title_align: ImVec2,
    pub child_window_rounding: f32,
    pub frame_padding: ImVec2,
    pub frame_rounding: f32,
    pub item_spacing: ImVec2,
    pub item_inner_spacing: ImVec2,
    pub touch_extra_padding: ImVec2,
    pub indent_spacing: f32,
    pub columns_min_spacing: f32,
    pub scrollbar_size: f32,
    pub scrollbar_rounding: f32,
    pub grab_min_size: f32,
    pub grab_rounding: f32,
    pub button_text_align: ImVec2,
    pub display_window_padding: ImVec2,
    pub display_safe_area_padding: ImVec2,
    pub anti_aliased_lines: bool,
    pub anti_aliased_shapes: bool,
    pub curve_tessellation_tol: f32,
    pub colors: ImColorsSave,
}

const CONFIG: &str = "
ImGuiStyleSave(
    alpha: 1.0,
    window_padding: (x: 8, y: 8),
    window_min_size: (x: 32, y: 32),
    window_rounding: 9.0,
    window_title_align: (x: 0.0, y: 0.5),
    child_window_rounding: 0.0,
    frame_padding: (x: 4, y: 3),
    frame_rounding: 0.0,
    item_spaciing: (x: 8, y: 4),
    item_inner_spacing: (x: 4, y: 4),
    touch_extra_padding: (x: 0, y: 0),
    indent_spacing: 21.0,
    columns_min_spacing: 6.0,
    scrollbar_size: 16,
    scrollbar_rounding: 9,
    grab_min_size: 10,
    grab_rounding: 0,
    button_text_align: (x: 0.5, y: 0.5),
    display_window_padding: (x: 22, y: 22),
    display_safe_area_padding: (x: 4, y: 4),
    anti_aliased_lines: true,
    anti_aliased_shapes: true,
    curve_tessellation_tol: 1.25,
    colors: (text: 4),
)";

#[test]
fn serde_with() {
    ron::de::from_str::<ImGuiStyleSave>(CONFIG).unwrap();
}

---- serde_with stdout ----
	thread 'serde_with' panicked at 'not yet implemented: IdDeserializer may only be used for identifiers', src/de/id.rs:224:8

It should be possible to write an automatic RON->JSON converter for cases when you have a .ron file and an app that only reads .json files.

Initial discussion on gitter: https://gitter.im/ron-rs/ron?at=598b80f551915d962af79f5d

See https://www.reddit.com/r/rust/comments/6sm6i2/rusty_object_notation/dlei3gp/

Basically, Rust clearly differentiates between integer types and floating points - there is no ambiguity. No reason for Ron to mix those types either.

The extensions page just says TODO.

In UNIC, we put auto-generated data tables into files we are calling Rust-Value (RSV), extensioned .rsv, which can contain basically any Rust expression (with all Rust requirements, like UTF-8 encoding), to be used inside the code using include!(<path-to-rsv>).

Examples:

• RSV table: https://github.com/behnam/rust-unic/blob/10696e882f2ebf027a162cfef987ac1053e94418/unic/ucd/age/src/tables/age_values.rsv
• Usage: https://github.com/behnam/rust-unic/blob/10696e882f2ebf027a162cfef987ac1053e94418/unic/ucd/age/src/age.rs#L79

What I want to note here is that RSV is more easier to use inside Rust code, being able to just include!()d. That's not exactly possible with RON, as the syntax is slightly different.

BUT, with a compile-time RON parser, it is possible to use RON in cases similar to UNIC's.

Just some thoughts, as we were considering to simplify data dumping process by using a serde-based solution, but couldn't find any existing tool.

https://github.com/typesafehub/config

Current errors are not very helpful since they pin no location on where they occur. Would be nice to have something like "expected A, got B", or "expected A at line XXX col YYY".

At this point in time there does not appear to be a pretty module (but there is to_string_pretty in the ser module):

A struct in Rust looks like this

struct Struct {
    x: f32,
    y: f32,
}

Serializer

The serializer uses the following representation:

Struct(x:3.0,y:2.0,)

Deserialization

The values are expected to come in comma separated and surrounded by ( and ). The name of the struct is optional.

https://hjson.org/

I could see include!("path") being useful, still looks in line with Rust syntax.
cc @torkleyy

rustc-serialize is now deprecated in favour of Serde, which hit 1.0

We should probably put the specification for the grammar, and any related stuff (like recommendations or common uses), in the repo itself instead of the wiki. This way, we can modify and discuss the spec in pull requests; and the spec will be protected from a random person deciding to hit the "edit" button.

between ( and [ of a newtype_struct wrapping a seq:

#[derive(Debug, Deserialize)]
struct Newtype(Vec<String>);

fn main() {
    let serialized = stringify!(
        Newtype([
            "a",
            "b",
            "c",
        ])
    );
    println!("{}", serialized);
    let deserialized: Result<Newtype, _> = ron::de::from_str(serialized);
    println!("{:#?}", deserialized);


    let serialized = r#"
        Newtype([
            "a",
            "b",
            "c",
        ])
    "#;
    println!("{}", serialized);
    let deserialized: Result<Newtype, _> = ron::de::from_str(serialized);
    println!("{:#?}", deserialized);
}

Newtype ( [ "a" , "b" , "c" , ] )
Err(
    Parser(
        ExpectedArray,
        Position {
            col: 10,
            line: 1
        }
    )
)

        Newtype([
            "a",
            "b",
            "c",
        ])
    
Ok(
    Newtype(
        [
            "a",
            "b",
            "c"
        ]
    )
)

(This is what I actually came here to report when I got sidetracked by the linguist+ronn history dive 😆)

Currently, they look like \u0007. In Rust however, it is ``\u{0007}`.

@kvark Should we change that?

And maybe it's a good idea to add octal and binary (0o... and 0b...) notations too.

Interesting quote from a reddit comment:

No specification for how to handle dates or times. By convention, we've all kind of settled on ISO 8601, but JSON just gives us a string really. If you work with some older web services that offer JSON, particularly .NET-based ones, you're likely to get bit by this.

Currently ASCII control characters like '\x07' are directly serialized. I expect them to be escaped like in Rust Code.

I think allowing that (as we currently do) is confusing; it provides no extra information, but makes it ambiguous whether it is a unit or an enum variant.

The current implementation of deserialize_any treats all numeric values as f64.

This causes issues when using crates such as serde-value as it will deserialize to the F64 variant preventing it from subsequently being able to be deserialized to an integer field, e.g.:

#[derive(Deserialize, Serialize)]
struct Config {
    value: u32
}

fn value_deserialize_ron() {
    let serialized = r#"(
        value: 5
    )"#;

    let value = ::ron::de::from_str::<::serde_value::Value>(serialized).unwrap();

    let config = value.deserialize_into::<Config>().expect("failed to deserialize from value");
}

This fails on the last line with the following error:

failed to deserialize from value: InvalidType(Float(5.0), "u32")

Both serde yaml:

fn value_deserialize_yaml() {
    let serialized = r#"
        value: 5
    "#;

    let value = ::serde_yaml::from_str::<::serde_value::Value>(serialized).unwrap();

    let config = value.deserialize_into::<Config>().expect("failed to deserialize from value");
}

And serde json:

fn value_deserialize_json() {
    let serialized = r#"{
        "value": 5
    }"#;

    let value = ::serde_json::from_str::<::serde_value::Value>(serialized).unwrap();

    let config = value.deserialize_into::<Config>().expect("failed to deserialize from value");
}

Are able to handle this case correctly, so I believe the issue is with how Ron parses these numeric values.

# Cargo.toml
ron = "0.2.1"

Given the following type:

pub enum FileOrMem {
    File(String),
    Memory,
}

ser::to_string produces the following RON:

            data: Node (
                location: Location (
                    file_or_mem: File(
                        "foo/bar"
                    ),
                    column: 1,
                    line: 1
                ),
                key: "Player",
                value: None,
                comment: None
            )

Which fails deserialization with:

Parser(ExpectedString, Position { col: 39, line: 19 })

The fix, IME, is to change the text to be inline. But this should be fixed in parsing IMO (what I mean is that this syntax is reasonable and should be accepted).

For the sake of completeness this error is not restricted to string literals on a new line.

It also occurs if there is a space between the ( and ".

It looks like RON is self-describing. It should be able to tell what type is in the data without relying on a Deserializer hint.

#[macro_use]
extern crate serde_derive;

extern crate ron;

#[derive(Deserialize, Debug)]
#[serde(untagged)]
enum Untagged {
    U8(u8),
    Bool(bool),
}

fn main() {
    println!("{}", ron::de::from_str::<u8>("99").unwrap());
    println!("{}", ron::de::from_str::<bool>("true").unwrap());

    // unhelpful panic :(
    ron::de::from_str::<Untagged>("true");
}

Are there plans to add (nestable) multiline comments?

https://github.com/lifthrasiir/cson

There are two kinds of unit types in Rust:

• ()
• struct Unit;

Serialization

The serializer uses the Rust representation, so () for () and Unit for Unit.

Deserialization

() is always a valid unit type and FooUnit is allowed for deserializing into named unit types.

http://www.hyperrealm.com/libconfig/libconfig_manual.html#Configuration-Files-1

Tuple structs look like this:

struct TupleStruct(f32, f32);

Serializer

TupleStruct(2.0,3.0,)

Deserializer

TupleStruct(2.0, 3.0) or (2.0, 3.0)

Unsure if this is a problem here, or ndarray, but considering I have no issues deserialising from json, yaml, messagepack etc to ndarray types, then I'm guessing here is a better place to start.

Using the Array3 type (which implements Deserialize if you enable the serde-1 feature) as an example, one can obtain the correct RON representation using to_string, but both from_reader and from_string yield an error.

A quick test:

let test = "(
    v: 1,
    dim: (1, 1, 1),
    data: [
        12.234,
    ],
)";

let conv: Array3<f64> = match ron::de::from_str(&test) {
    Ok(x) => x,
    Err(e) => {
        println!("{}", e);
    },
};

sees the following:

invalid type: byte array, expected "v", "dim", or "data"

Is this me configuring things incorrectly or has something been overlooked in this instance?

Newtype structs wrap an existing type:

struct NewType(i32);

I'm not sure here. Should it be NewType(3) with the name being optional? Or just 3?

https://github.com/hauleth/ucl-rs (my bindings to libucl written in C, maybe I rewrite it to Rust when I will have spare time).

I cannot find any other contact to you.

Enums look like

enum Enum {
    A,
    B(i32),
    C(f32, f32),
    D { x: f32, y: f32 },
}

Proposed strategy: Only serialize / deserialize the variant name and use the format we have for unit types, newtype wrappers, tuple structs and structs, respectively.

If the toplevel RON structure is an array, it needs matching [ and ] brackets in the beginning and end of the file, correct? What if I want to make an incremental log file that is only appended to and that parses as a valid RON file containing an array of log entry values after each write? I'd have to modify the existing file to remove the final ] before each new write.

There's JSON Lines for the JSON version of an incremental file. Maybe RON could have some built-in support?

A very hacky and nasty approach would be to simply let you omit the final closing ] if the toplevel syntax element is an array.

My recommendation is to use the # character to begin a single-line comment.

These can be a killer feature in any human-readable data format.

I'd like the ability to enable struct_names (via PrettyConfig) in

Current proposal:

• require trailing comma for maps and non-tuple structs (which have named fields)
• optional comma for tuples and tuple structs

https://github.com/hashicorp/hcl

So I've been tossing an idea for a data format around in my head for a while now, and this seems like a good application so I thought I would throw it out.

Basically the data format is roughly this: xml in json-syntax. The idea is to have a json/yaml-like syntax but with attributes. So you could write something like this:

foo @{x: 42, y: 72, z}:
    a: "this is a key: string"

@dec_attr: "This is a decorating bar"
bar: "this is bar"

The basic idea is that attribute maps denoted by @ can be placed either after a key or above a key. After a key they must be in the form @{ ... } and above the key they must be in the form @key = value.

The basic idea is to make it easy to have keys with attributes associated with them -- such as types, generics, compiler flags, etc. This kind of language could be used for writing type systems, as a new programming langauge, or almost anything! If you think about it, programming languages are pretty much just variable names with some attributes (as well as some logic handlers... which would not be included in this spec).

Maybe significant white-space is not desired, or requiring "" around keys is desired. You could still use much the same syntax:

{
  "foo" @{"x": 42, "y": 72, "z": true}: {
    "a": "this is a key: string",
  },
  @"dec_attr": "This is decorating bar",  # <-- note the `,` is necessary
  "bar": "this is bar",
}

Also note that {z} is interpreted {z: true} to make passing flags easier. That could also be removed.

One advantage of this system is a clear distinction between the user's logic and the compiler or type-system's logic. The compiler is pretty much only concerned with the attributes (which the user uses to communicate with the compiler). So everything in an attribute is a compiler/language directive, and everything outside an attribute is a user-defined name/value/type.

Anyway, just something I've been tossing around in my head, thought you might like it for some food for thought!

This distinction allows .....(us / you?)..... to solve the biggest problem with JSON.

Possible things to configure:

• type of indentation (N spaces, tab, etc)
• type of newlines
• making indentation for tuples

what else?

If I have a struct like

#[derive(Debug, Clone, Deserialize)]
pub struct Settings {
    pub font: Option<PathBuf>, // <- optional field
    pub other_value: u8,
}

I can omit the field in .ron file to get None:

(
    other_value: 0,
)

But I'm forced to write Some() around the actual value which clutters up the config:

(
    font: Some("OpenSans-Regular.ttf"), // :(
    // font: "OpenSans-Regular.ttf", // I want this
    other_value: 0,
)

Seems like an ergonomic problem to me :(

See https://gitter.im/ron-rs/ron?at=59d228bd614889d47565733d

After a particular depth, don't insert newlines any more, e.g. (from @pyfisch custom format):

┌ Display List
│  ├─ ClipScrollNodes
│  │  └─ ClipScrollNode { id: Some(Clip(0, PipelineId(0, 1))), parent_index: ClipScrollNodeIndex(0), clip: ClippingRegion::Empty, content_rect: TypedRect(0.0×0.0 at (0.0,0.0)), node_type: ScrollFrame(ScriptAndInputEvents) }
│  ├─ Items
│  │  ├─ PushStackingContext(StackingContext at TypedRect(1024px×1077.2333333333333px at (0px,0px)) with overflow TypedRect(1024px×1077.2333333333333px at (0px,0px)): StackingContextId(0)) StackingContext: StackingContextId(0) ClippingAndScrolling { scrolling: ClipScrollNodeIndex(0), clipping: None }
│  │  ├─ SolidColor rgba(0, 0, 0, 0) @ TypedRect(1024px×1077.2333333333333px at (0px,0px)) Rect(TypedRect(1024.0×1077.2333 at (0.0,0.0))) StackingContext: StackingContextId(0) ClippingAndScrolling { scrolling: ClipScrollNodeIndex(0), clipping: None }