divnix / dmerge Goto Github PK

A mini merge DSL for data overlays

License: MIT License

Nix 99.49% Shell 0.51%

dmerge's Introduction

Dmerge

A mini merge DSL for data overlays.

Dmerge is a lightweight alternative to the NixOS module system to wrangle data. It aims to give you alternative semantics, currently not available in the ecosystem, when you need them.

Semantics

Simple semantics: merge :: lhs -> rhs -> final
Monotonistic on the Dataspine
Expressive Array Merge Decorations
Typed-by-Precedent

Simple semantics

# nix repl

> :lf github:divnix/dmerge

> :p merge { foo = "bar"; } { foo = "baz"; }
{ foo = "baz"; }

> :p merge { foo = [1]; } { foo = append [2]; }
{ foo = [ 1 2 ]; }

> :p merge { foo = [1]; } { foo = prepend [2]; }
{ foo = [ 2 1 ]; }

# update [idx] updates idx on lhs
> :p merge { foo = [1]; } { foo = update [0] [2]; }
{ foo = [ 2 ]; }

# recurses the arrays and attribute sets
> :p merge { foo = [{egg = {color = "yellow";};}]; } { foo = update [0] [{egg = {color = "green";};}]; }
{ foo = [ { egg = { color = "green"; }; } ]; }

# supports associative array merges
> :p merge { people = [{name = "bert"; age = 42;}]; } {people = updateOn "name" [{name = "bert"; age = 43;}]; }
{ people = [ { age = 43; name = "bert"; } ]; }

# chaining
> WithMichi = chainable {michelangelo = { age = 548; };}
> WithRemi = chainable {rembrandt = { age = 417; };}
> WithLeo = chainable {davinci = { age = 571; };}
> mkParty = chainMerge
> :p mkParty WithMichi WithRemi WithLeo {me = {age = 35;};}
{ davinci = { age = 571; }; me = { age = 35; }; michelangelo = { age = 548; }; rembrandt = { age = 417; }; }

Monotonistic on the Dataspine

Dmerge never destroys an Attribute Set or an Array. Unlike simple values, they are considered the dataspine and must be protected.

If you have a use case for destroying an Attribute Set or Array or plainly overriding an Array, consider better factorizing your left hand side, instead. It's a symptom of poor factorization.

Expressive Array Merge Decorations

In the above examples prooving its simplicity, you already got to know the full instruction set of Dmerge to specify the array merge strategy on the right hand side: append, prepend, update & updateOn.

If you don't control the right hand side, but know it's structure, you can decorate it:

merge lhs (decorate rhs { people = updateOn "name";})

Typed by Precedent

You can't modify the type of a left hand side node or leave in the right hand side.

Tests

Check out the tests for more examples.

Implementor's Note

This repo has no flake.lock. Chose the dependencies you see fit; assert compatibilty — chances are high, you'll be fine, though.

Example Use:

# flake.nix
{
  inputs.nixpkgs.url = "github:nixos/nixpkgs";
  inputs.dmerge = {
    url = "github:divnix/dmerge";
    inputs.nixpkgs.follows = "nixpkgs";
    inputs.namaka.follows = ""; # testing only dependency
  };

  /* ... */
}

Acknowledgements

Haumea for help me chore
Namaka for testing-made-easy (tm)
Cocogitto for taking the guesswork out of releases
nix-quick-install-action for making nix in CI a sure quick thing

dmerge's People

Contributors

Stargazers

Watchers

Forkers

ilkecan

dmerge's Issues

Inconsistent behavior on new keys in RHS

Defining new keys works as expected if you don't use merge functions in the RHS:

merge {} {
  new = [];
  nested.new = [];
}

{
  "nested": {
    "new": []
  },
  "new": []
}

But if you use a merge function it fails:

merge {} {
  new = append [];
}

error: evaluation aborted with the following error message: '
       a fresh right-hand-side cannot be an array merge function
       at 'new':
         - rhs: lambda @ …

Ok, understandable. Unexpectedly it works if the new key is nested, although the result is the append lambda itself. This is probably not what the user wants:

merge {} {
  nested.new = append [];
}

{ nested = { new = <LAMBDA>; }; }

This seem inconsistent. Defining a new key with a merge function in the RHS should either always fail with a helpful message or always succeed.

I tend to prefer the latter because that would be consistent with the behavior we observe if no merge functions are used in the RHS (see first snippet).

The UX benefit would be that it always just works™.

One drawback could be that you can make less guarantees about the outcome from looking at the LHS. However, we can already not make any guarantees about which new keys will exist in the result since that depends on whether the new keys in the RHS use merge functions or not, and whether they are nested or not, which the LHS does not know. But the benefits of the data spine would still hold if data-merge accepted both scenarios in the RHS.

The real life problem is that you may not always know which keys exist in the LHS if you get it from elsewhere, like when writing a library. Currently it is necessary to pre-create the new keys in the LHS first to allow merging in possibly new keys with merge functions from the RHS (runnable example):

{
  inputs.data-merge.url = "github:divnix/data-merge";

  outputs = { nixpkgs, data-merge, ... }: {
    result = with data-merge; let
      lhs = {}; # we may not know this value
      rhs = { new = append []; };
      # defeats the purpose of data-merge
      prepopulatedLhs = nixpkgs.lib.recursiveUpdate lhs {
        new = lhs.new or [];
      };
    in merge prepopulatedLhs rhs; # ok
    # in merge lhs rhs; # fails
  };
}

We need `dm.decorate` to decorate a `rhs` conveniently without needing access to it's source

rhs' = dm.decorate rhs { path.to.list = dm.append;};

List updating is broken on latest commit

nix-repl> dm = inputs.data-merge 

nix-repl> dm.merge { a = [ 6 ]; } { a = dm.update [ 0 ] [ 7 ]; }
{ a = [ ... ]; }

nix-repl> (dm.merge { a = [ 6 ]; } { a = dm.update [ 0 ] [ 7 ]; }).a
[ { ... } ]

The first item in that list should be 7.

cc @blaggacao this is resulting in syntax errors in nomad jobs built in cardano-world.