Git Product home page Git Product logo

walk's Introduction

Walk

Build Status Go Report Card Latest Version

walk is a fast, general purpose, graph based build and task execution utility.

Heavily inspired by make and redo.

Features

  • Fast parallel execution.
  • Graph based dependency management.
  • Maximum composability with existing UNIX tooling.
  • Describe targets and their dependencies as simple executables.
  • Universal execution; execute walk from any directory.

Installation

Using Go 1.7+:

$ go get -u github.com/ejholmes/walk

Or grab the latest release from https://github.com/ejholmes/walk/releases.

Usage

walk is built on top of a very simple concept; when you want to build a target, walk executes a file called Walkfile to determine:

  1. What other targets the given target depends on.
  2. How to build the target.

For example, if you wanted to build a program called prog from main.c and parse.c, you might write a Walkfile like this:

#!/bin/bash

# The first argument is the "phase", which will either be `deps` or `exec`. In
# the `deps` phase, the Walkfile should print the name of the targets that this
# target depends on.
phase=$1

# The second argument is the name of the target, like `prog`, `parse.o`, etc.
target=$2

case $target in
  prog)
    case $phase in
      # Prog depends on the object files we'll build from source. We simply
      # print each dependency on a single line.
      deps)
        echo main.o
        echo parse.o
        ;;
      exec) exec gcc -Wall -o $target $($0 deps $target) ;;
    esac ;;

  # A generic recipe for building a .o file from a corresponding .c file.
  *.o)
    case $phase in
      deps) echo ${target//.o/.c} ;;
      exec) exec gcc -Wall -o $target -c $($0 deps $target) ;;
    esac ;;

  # When invoking walk(1) without any arguments, it defaults to a target called
  # `all`.
  all)
    case $phase in
      deps) echo prog ;;
    esac ;;

  # In general, it's good practice to include a fallback rule like this, in
  # case someone tries to build a target that we don't know how to build (or
  # someone makes a typo).
  *.c|*.h) ;; # static files
  *) >&2 echo "No rule for target \"$target\"" && exit 1 ;;
esac

When you execute walk all, the following happens internally:

  1. walk resolves all of the dependencies, and builds a graph:

    $ Walkfile deps all
    prog
    $ Walkfile deps prog
    parse.o
    main.o
    $ Walkfile deps parse.o
    parse.c
    $ Walkfile deps main.o
    main.c
    $ Walkfile deps parse.c
    $ Walkfile deps main.c
  2. walk executes all of the targets, starting with dependencies:

    $ Walkfile exec parse.c
    $ Walkfile exec main.c
    $ Walkfile exec main.o
    $ Walkfile exec parse.o
    $ Walkfile exec prog
    $ Walkfile exec all

Ultimately, all of our targets end up getting invoked, and prog is built:

$ walk
ok	main.c
ok	parse.c
ok	parse.o
ok	main.o
ok	prog
ok	all

We can print the dependency graph to verify that our dependency chain is what we expect:

$ walk -p dot
digraph {
  "(root)" -> "all"
  "all" -> "prog"
  "prog" -> "main.o"
  "prog" -> "parse.o"
  "parse.o" -> "parse.c"
  "main.o" -> "main.c"
}

And that's it. Wait, that's it? That's it. walk is quite simply, just syntactic sugar over executing a binary as a graph.

See also man walk.

walk's People

Contributors

ejholmes avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

Forkers

dkrikun

walk's Issues

Better getting started help text

Right now, If I cd to an empty directory, and execute walk, this is what I see:

dir $ walk
dir $

No output. walk is doing the right thing, since the intended behavior of a dependency in a directory without a Walkfile is to assume that the dependency is a static file, or a noop dep.

However, in the above case, where I'm executing a root target that has no Walkfile, it would be nice to output some helpful message on stderr:

dir $ walk
WARNING: No Walkfile found for target "all"
See https://ejholmes.github.io/walk#getting-started

Ability to use a single Walkfile

walk currently forces the user to split the dependency/build rules in multiple Walkfile files.

Could you consider, as a different or alternative mode of operation, that there is one Walkfile per project ?

If I understood correctly, it would solve #20 as a side effect.

Cheers !

Ability to escape slashes

Let's say you want a Walkfile that installs homebrew taps:

#!/bin/bash

phase=$1
target=$2

case $target in
  *.tap)
    case $phase in
      exec) brew tap ${target%.tap} ;;
    esac ;;
esac

Now, I want to do something like this:

walk caskroom/cask.tap

walk is going to look in a directory called caskroom and execute the cask.tap target. Maybe we should allow escaping slashes to indicate that it's not really a directory:

walk caskroom\/cask.tap

More intelligent default semaphore

By default, the number of targets that are executed in parallel is unlimited. This should probably, at the very least, be limited based on the number of file descriptors in ulimits.

Parallelize planning

walk/plan.go

Lines 128 to 134 in 11c4e90

// TODO(ejholmes): Accept a semaphore and parallelize this. No
// need to perform this serially.
dep, err := p.newTarget(ctx, d)
if err != nil {
return err
}
p.graph.Connect(t, dep)

Sandboxing desirable?

I think it would be nice to have the (perhaps global) option to exec the Walkfile in some kind of sandbox

Bug: Depending on absolute path doesn't work

If you're dependency includes an absolute path, like /src/foo.c, walk(1) will expand this to a relative path.

Anything that is already an absolute path should be used as is.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.