maxmcd / bramble Goto Github PK

• the lockfile is not updated if the derivation has been built
• the derivation doesn't contain the hash, but it's not available until after we build

do we need to fix these things? will they cause problems?

NixOS/nix@276a40b

I would like to start an open-ended discussion about one of the difficulties I see with 'in-the-wild' nix configurations, so folks can share their thoughts and experiences without necessarily 'solutioning an issue'. If you see issues as reasonable for open-ended discussion, I'll close this and open a new one with the specific concerns I have in mind.

Complete the functionality that would allow bramble to use starlark as a build script. Once this issue is complete we should be able to build this derivation:

def run_busybox():
    bb = busybox()
    # whoami is now available to run
    bb.cmd("whoami").print()


def busybox():
    # download the executable, this is the only way you are allowed
    # to use the network during builds
    download = derivation(
        name="busybox_download",
        builder="fetch_url",
        env={
            "url": "https://brmbl.s3.amazonaws.com/busybox-x86_64.tar.gz",
            "hash": "2ae410370b8e9113968ffa6e52f38eea7f17df5f436bd6a69cc41c6ca01541a1",
        },
    )

    return derivation(
        name="busybox",
        builder=build_busybox,
        deps=[download],
    )


def build_busybox():
    os.mkdir("$out/bin") # using the builtin os module

    # move the busybox executable into the output for this build
    cmd("$busybox_download/busybox-x86_64",
        "cp",
        "$busybox-x86_64 $out/bin/busybox").wait()
    commands = []
    commands_started = False

    # run the busybox help text to extract supported commands
    for line in cmd("$out/bin/busybox").output:
        if commands_started:
            for name in line.strip().split(","):
                commands.push(name.strip())
        if "Currently defined functions" in line:# the builtin os module
            commands_started = True

    # move into the output directory so that symlinks are relative
    os.cd("$out/bin")

    cmd("./busybox ln -s busybox ln").wait()
    # link each command
    for cmd in commands:
        cmd("./ln -s busybox %s" % cmd).wait()

So that previous versions can be pulled and run as needed

Most of the desired initial features are now complete, but there are a lot of gnarly edges. Seems it would make sense to do a cleanup and stability sprint. Things to cover:

• variable expansion is complex, figure out how env vars and $bramble_path work with each other
• implement garbage collection
• improve docs
  • no incorrect code in examples (implement tests in docs)
  • update various notes files as well
  • rewrite readme
  • add tons of examples
• test that derivations can actually rebuild when needed
• add lockfile and url hashes to lockfile
• write lots of tests in bramble
• start supporting different systems, get osx working
• lazy derivation execution
  • cmd call triggers supporting derivation builds
• multiple outputs created and working
• can tests be run in any file? rust can do it, check zig?
• debugger, open shell on excution error google/starlark-go#304
• parallel building
• build a docker image (stability and cleanup huh?)
• actually make derivations and cmd impossible to run at the top level
• implement output path replacement for self-referential outputs
• no automatic variable expansion in input functions
• fork starlark-go and reimplement load function

I wonder what are the (dis)advantages of Bramble compared to nix (which is itself mentioned in the readme but not elaborated).

Parsing will just result in derivations to build and then other derivations that could have been built along the way. Instead of using the single Bramble struct for everything, decouple the functionality so that code is parsed and then json/derivations are outputted.

Now that they're derivations that project/build boundary makes this tricky.

It's currently possible to build things and then visit the build output directory to run software, but it's not really possible to do much else. There used to be functionality to do this, but because Bramble expects to allow people to trivially run software pulled from remote repositories it feels important to create an opinionated default way to run things that won't allow these programs to do dangerous things to the host machine.

Some open questions:

• Is this something that is defined in the build files?
• Is it a command you can run?
• Are you allowed to run things outside of a project?
• If we run things in a sandbox how do you define what folders the program should have access to?

The nested docker containers are a performance hit. Seems like we could have a similar level of isolation using a single docker container and a more careful use of linux permissions.