Inclusion of other configs about ron HOT 14 CLOSED

Specifying file inclusions in a serialization format ties it to the filesystem and introduces new and exciting opportunities for errors when deserializing data (your in-memory RON buffer can reference files that don't exist, that your application shouldn't read, or that contain include cycles). It also interacts with serialization: if deserialization dereferences file inclusions, a roundtrip of deserialize -> serialize will necessarily inline the entire included file. You could add "serialize but assume this subtree of the object graph is already serialized in a file with a given name" to the API, but this is significant added complexity.

Every increase in scope for the data format is a pitfall for interoperating implementations, and in my opinion we should avoid these if at all possible. In situations where file inclusions are desirable (like a program running atop a filesystem, e.g. on unix, that wants to share portions of its configuration between 'base' and 'customized' files), these can be specified inside the data structures serialized to RON, and interpreted and loaded by a wrapper library or the program. I think RON itself should not be limited to applications where a filesystem makes sense, nor complicate its interface and implementation by interacting with the filesystem.

Variable expansion, as noted by @BernhardPosselt, gives rise to similar concerns. RON should learn from the mistakes of XML, not repeat them.

from ron.

This sounds rather useful, I agree, but I also want to stress that this can be strictly added after we have the basic format implemented.

from ron.

I'm losing track of what's going on here. First of all, constants discussion doesn't seem to be related to inclusion, so it needs to be moved over into a dedicated issue. Secondly, for the gravity case, I assume you want to refer to the constant from other parts of the config?

E.g. somewhere at the top you'd have

gravity = 9.8

Then below, you could have

some_force = 0.5 * @globals.gravity

@torkleyy is that what you have in mind?

from ron.

BTW, there are similar things in XML like XInclude and External Entities. They are exploited in attacks (XXE) and a huge headache everytime you deal with user input. Basically you turn off those features everytime and pray.

from ron.

from ron.

Sounds like a cool feature to me! @torkleyy I think we already do allow constants? like

Foo(
    bar: 1,
)

The 1 is a constant. Aren't constants more or less the whole purpose of Ron?

Alternatively you might mean allowing Rust constants to be set in Ron, which isn't actually possible thanks to the definition of a Rust constant. The closest we could get to that would be allowing a static variable to be modified by Ron but that would require our users to whip out something unsafe. static variables are generally a bad design pattern anyways imo.

from ron.

Include would be useful, but also limiting if you cannot change what is brought in. I'd really say take a look at how HOCON handles includes, where/how they are brought in, how to change values, and how to reference other values somewhere else in the tree (one-definition-rule in my opinion). Would be a hugely useful feature to add though!

from ron.

from ron.

With our current setup we could do

Config(
    gravity: 9.8,
)

which doesn't require us to be creative about how Rust would interpret the value.

from ron.

@kvark

I assume you want to refer to the constant from other parts of the config?

Yes.

Then below, you could have

some_force = 0.5 * @globals.gravity

Well, partially, but I don't think we support expressions. So just

ParticlyComponent(
    force: @globals.gravity
)

Note that "constants" aren't limited to literals, can also just be "structs" in that config which you define once and use in multiple places. I think this is indeed somehow related with inclusion, because we also have to consider if purely inserting content from another file is enough / useful.

from ron.

Note that "constants" aren't limited to literals, can also just be "structs" in that config which you define once and use in multiple places. I think this is indeed somehow related with inclusion, because we also have to consider if purely inserting content from another file is enough / useful.

And in hocon when you refer to another literal (or include another file at a place) you can then define those values again to override what they were but only in your specific location. This is an absolutely useful feature to swipe from hocon. :-)

from ron.

I agree with @sp3d, inclusion can easily be added by a wrapper for those that need such things. Don't complicate the format with things that aren't really needed imho.

from ron.

Might I suggest formalizing tiers of support. So when people go to implement these RON standards they can document to the user what level of support their code is at. Allowing RON to cover more then just simplistic configurations. something like: -tier 1: cover all the basic stuff structs, maps, arrays, etc. -tier 2: inclusion(this issue), references, etc. -tier 3: ... that way you can have this feature without bogging down implementations of simpler tier 1 support. If json had tierd support they could have stayed relevant without fragmenting into Hjson and other derivatives. supporting through higher tiers things like: trailing commas and white-space, multi-line strings, literal strings, references, comments ,etc.

from ron.

I think we should not embed this into the format and leave it up to the user to implement such mechanisms.

from ron.