stepchowfun / toast Goto Github PK

Description
It seems that PATH is overridden when using su. This is not the case when using docker directly.

Instructions to reproduce the bug

$ cat toast.yml 
image: eclipse-temurin:11-jdk-focal
tasks:
  java_version:
    command: java -version

$ toast
[INFO] Ready to run 1 task: java_version.
[INFO] Running task java_version…
bash: java: command not found

Example correct output using docker directly:

$ docker run -t eclipse-temurin:11-jdk-focal java -version
openjdk version "11.0.17" 2022-10-18
OpenJDK Runtime Environment Temurin-11.0.17+8 (build 11.0.17+8)
OpenJDK 64-Bit Server VM Temurin-11.0.17+8 (build 11.0.17+8, mixed mode, sharing)

I suspect this is because su is invoked, and that the PATH environment variable is then overridden.

Environment information:

• Toast version: 0.45.5
• OS: Linux 5.15.76-1-MANJARO #1 SMP PREEMPT Sat Oct 29 14:22:16 UTC 2022 x86_64 GNU/Linux

Is there any way to fix this?

Thanks for an interesting and good project!

Description
When using the --shell or -s parameter to drop into a shell in the container after all the tasks have run, the "Unable to canonicalize the source directory. Reason: No such file or directory (os error 2)" error rises and execution stops.

Instructions to reproduce the bug
run toast with the -s parameter, it happens with every Toastfile, for example

image: debian:stable-slim

tasks:
  test:
    command: |
      echo Hello

This is what happens:

deadda7a@rainbow ~ % toast -s -f test.toast
[INFO] Ready to run 1 task: test.
[INFO] Running task test…
Hello
[INFO] Preparing a shell…
[ERROR] Unable to canonicalize the source directory. Reason: No such file or directory (os error 2)

Environment information:

• Toast version: 0.45.4
• OS: Debian Sid

Additional context
I put Debian Sid in the OS info because I use it locally on my dev machine, but the same issue also exists on Debian Stable (11 at the time being)

Maybe this is related to issue #245?

Also thank you for this awesome tool :)

Description
I was going through the guide when I got the error "Unable to pull image" without any details.
Turns out I was just missing sudo...
However, it might be helpful for users who are new to docker, toast or Linux to get a more helpful error message.

Here is the output I got.

$ toast
[INFO] Ready to run 1 task: greet.
[INFO] Running task greet…
[ERROR] Unable to pull image. Details:

I can run toast without sudo on osx so there is no issue there.

System
Arch Linux 5.1.6
Docker 18.09.6-ce
rustc 1.34.2
Toast 0.24.0

(Side Note, sorry for all the issues I'm submitting...I"m in the middle of refactoring a large number of Toastfiles so it's on my mind 😄 I'm also a big fan of Toast!)

Description

Add the ability to print out the task schedule in order.

Alternatives considered

None

Additional context

While doing some larger refactoring of my toastfiles I found myself working through the dependencies of my tasks to make sure the order of execution came out right, and thus shared layers between the various components. The only way I could do this was to run trough the toastfile though by actually running the tasks.

Adding a simple --print-schedule would solve this and allow one to work through the order and ensure dependencies are correct before running the tasks.

I have a working implementation of this read for a PR if interested.

Since Toast uses Docker bind mounts to bring files into a container, tasks with mount_paths will fail as the remote Docker engine specified by the DOCKER_HOST environment variable won't have those paths:

[ERROR] Unable to create container.
Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /Users/dev/work/incubator/container-workflow-test/foo.html

Not sure what options are for supporting this use case (or if it's desired at all), but thought it would be worth mentioning.

Description

Adding a PowerShell installation script would make installation easier for windows users over the current very manual installation.

I have some experience with PowerShell but I have never written something like this before. I've made a first draft so if this issue gets a green flag I can start working on a pull request however I would really appreciate it if someone more experienced than me in PowerShell would review this :)

Hello,

I've made a mental note to use this project for a while now. And just as I was ready to use it, I couldn't find it! I searched through my browsing history to no avail. Searching "container" or "docker" in https://lib.rs does not show Toast. Finally, I found toast again by luck while browsing crates.io.

I'm not 100% sure why this is happening, but I suspect it could be because of a lack of project keywords. Perhaps adding keywords "container", "docker", and "cli" might help discoverability.

Description
I'm not entirely sure if it's already possible, but I couldn't find any evidence of it, but it would be great to have container linking and running services.
For example, for running tests that require a database or external cache, it's very useful to be able to have a long-running task run alongside another.
It's also useful for running integration tests where one service communicates with the other.

Alternatives considered
I've considered two alternatives:

• Docker Compose: This is nice and very useful, but it does not have the DAG like toast does. Neither is the caching on Docker Compose good enough for complex workflows, particularly in CI.
• Running parallel toast jobs: Running another toast job alongside one, but linking them with a virtual network so the applications/services can talk to each other.

I find that more often than not, the path within container is not the same as the path outside, and currently toast does not support it.

First, thank you for toast I love it!

Description

When /tmp is mounted as tmpfs (which is the default for certain major Linux distributions [such as Fedora]), using output_paths will fail with an error:

[ERROR] Unable to move file /tmp/.tmpFH7oiU/data to destination /home/foo/bar/baz. Reason: Invalid cross-device link (os error 18)

Instructions to reproduce the bug

• Mount /tmp as tmpfs
• Run any toastfile with an output_paths

System information:

• OS: Fedora 30
• Docker version: Docker version 18.06.3, build d7080c1
• Toast version: Toast 0.21.0

Additional context

From the docker cp help page:

It is not possible to copy certain system files such as resources under /proc, /sys, /dev,
tmpfs, and mounts created by the user in the container.  However, you can still copy such
files by manually running tar in docker exec.  For example (consider SRC_PATH and
DEST_PATH are directories):

    $ docker exec foo tar Ccf $(dirname SRC_PATH) - $(basename SRC_PATH) | tar Cxf DEST_PATH -

or

    $ tar Ccf $(dirname SRC_PATH) - $(basename SRC_PATH) | docker exec -i foo tar Cxf DEST_PATH -

Description
Each path which is configured on mount_paths section will result in a not usable path for Docker on Windows.
The problem seams to be, that toast uses the function canonicalize to create the absolute source dir which produces a extended length path on Windows which Docker cannot interpret.
The Result of this is a error message like following from Docker:
Error response from daemon: \\?\C:\Users\Testuser\Downloads%!(EXTRA string=is not a valid Windows path)

Instructions to reproduce the bug
Configure a mount_path in toast.yml and run toast.

Environment information:

• Toast version: v0.46.0
• OS: Windows

Description
Is it possible to create a named image of a task result?

Description

When exposing ports without setting cache: false, there is an error.

[ERROR] Unable to parse file /vagrant/temp/toast.yml. Reason: Task server exposes ports but does not disable caching. To fix this, set cache: false for this task.

Instead of showing an error, why not disable the cache?

Alternatives considered

docker-compose... 😄

Additional context

I was testing with this toastfile:

image: dannyben/alpine-ruby

tasks:
  setup:
    command: gem install httpme

  server:
    # cache: false 
    dependencies: [setup]
    ports: ["3000:3000"]
    input_paths: [docroot]
    command: httpme 

Description
Trying to set the input_paths to use a directory relative to above where toast.yml lives.

Instructions to reproduce the bug
A clear explanation of how to reproduce the bug.

System information:

• OS: macOS Mojave 10.14.5
• Docker version: 18.09.2
• Toast version: latest

Additional context
I'm trying to have multiple toast files for multiple containers, 1 for the server, 1 for the db. I figured I'd organize the toast files into their own directory to keep things simple and for the server need to copy over the source files.

Description

Github's scanner doesn't recognize your LICENSE.md.
The right sidebar just has License [ View license](https://github.com/stepchowfun/toast/blob/main/LICENSE.md).
For a moment I feared someone had written yet another license, but it seems LICENSE.md looks like it's MIT.

You might want to copy the file more byte-identically, maybe Github is fooled by something like word-wrapping.

Description
A clear description of what you want to happen.
One of the benefits of containerization of dev envs is being able to run them in a platform agnostic way, so it'd be cool if toast could run on Windows

Alternatives considered
A clear description of any alternative solutions or features you've considered.

Additional context
Add any other context about the feature request here.
I'd love to help make this happen.

If I use mount_paths and start toast with --shell I get empty files and directories. With this toast file:

image: ubuntu:16.04
default: mount
tasks:
  mount:
    cache: false
    mount_paths:
      - foo
    command: |
      cat foo

and:

echo "test" > foo

I get this problem:

luogni@luogni-XPS-13-9350:~/Temp/a$ toast
[INFO] Ready to run 1 task: mount.
[INFO] Running task mount…
test
luogni@luogni-XPS-13-9350:~/Temp/a$ toast --shell
[INFO] Ready to run 1 task: mount.
[INFO] Running task mount…
test
[INFO] Preparing a shell…
root@efe4bc98ca20:/scratch# ls -al
total 8
drwxrwxrwx 2 root root 4096 Jan  1  1970 .
drwxr-xr-x 1 root root 4096 Jun 20 12:20 ..
-rwxr-xr-x 1 root root    0 Jun 20 12:20 foo
root@efe4bc98ca20:/scratch# cat foo
root@efe4bc98ca20:/scratch# 

Description
I've been looking at potentially using a multi-stage Dockerfile to be able to reuse a lot of the existing configuration we have in the Dockerfiles used to create production Docker images, and make it possible to use that when running tests in the developer environment.

Managing big multi-stage Docker files is however not easy, and having it structured in YAML - like Toast does - would be an improvement, but in order for Toast to work in this case we would need a way of generating a Docker image from a specific task with an image ID we would know.

I guess this, to put it simply, is a request to add the equivalent of the -t/--tag flag from the docker build command. Does that sound like a reasonable addition?

Alternatives considered
As stated above the alternative would be to manage a big multi-stage Dockerfile, which would work, but means missing out on the features Toast brings.

Additional context
Add any other context about the feature request here.

Description
It would be nice if Toast had a man page, as most big projects have them.

Alternatives considered
No man page :p.

Additional context
I'd be willing to set this up myself if needed. I'd probably use Asciidoctor, as it's what I'm most familiar with for writing man pages, but I'm wondering if requiring that as a build dependency would be fine by this project's maintainers.

Description
Access private repositories via ssh by utilizing ssh agent forwarding. This would prevent the need to share this kind of secret with toast. This feature has been implemented in docker with buildkit and is described in the following article.

https://medium.com/@tonistiigi/build-secrets-and-ssh-forwarding-in-docker-18-09-ae8161d066

Alternatives considered
I'd be open to any alternatives that negate the need to copy a private key into a docker image that is ultimately cached.

Additional context
This is primarily a security feature for organizations that depend heavily on ssh during builds.

Description
Add an option to toastfile so that we can add arbitrary extra docker arguments

Additional context
I find that I cannot add --devices and --privileged etc. with current toastfile definition. But they are needed for example when we want to use GPU and other devices within the container. Adding an option to support any docker arguments can be a general solution to solve such problems.

Description
We currently use toast to run some codegen inside a container.

The codegen reads in a lot of files as input, but spits out only one or two files as output.

When the codegen is done, copying the output paths out is fast. Then toast spends an order of magnitude more time on the "Committing container..." step

Is there a way we could add an option skip that step?

Alternatives considered
I also wonder if Toast could automatically skip the container commit if it can figure out that it won't be used. But an option would also be fine.

Additional context
None, thanks for a great tool!

Description

One task A creates a directory in a container, then a later task B which depends on A, when using mount_paths, can remove that directory. I've tested and the same does not happen with input_paths.

Instructions to reproduce the bug

Here is my Toast file. The package.json contains Vite and some other stuff:

image: node:20
command_prefix: set -euxo pipefail
location: /scratch
tasks:
  deps:
    input_paths:
    - package.json
    - package-lock.json
    command: |
      npm install
  dev:
    cache: false
    dependencies:
    - deps
    mount_paths:
    - .
    ports:
    - 5173:5173
    command: |
      npm run dev

When trying to use toast dev, the command fails (vite is not found), and getting a shell into the container I found that the node_modules directory that had been created by the deps task was gone. Replacing mount_paths with input_paths in the above example, changing nothing else, fixes the problem (the node_modules dir exists and vite etc can be used).

Environment information:

$ toast -v
Toast 0.47.5

$ sw_vers
ProductName:	macOS
ProductVersion:	11.7.8
BuildVersion:	20G1351

Might this be because mounting . of the host to /scratch inside the container overrides all existing contents in /scratch? This would be a surprising difference between the behaviors of mount_paths and input_paths ("for each file in this directory, copy the file into the container") which otherwise look and work the same. Anyway, if this behavior is intended, then feel free to close the issue, thanks!

Is it possible to add a global environment parameter defined for all tasks? For example:

image: ubuntu
default: release
environment:
  RELEASE: 1.4.40

tasks:
  download:
    command: wget link_to_$RELEASE

  build:
    command: tar xzf path_to_$RELEASE.tar.gz

It would be nice if a user could run docker commands from the tasks in toast.

Specific commands like:

docker build
docker run

To do this you may need to add a switch to toast to allow privileged mode and to mount the docker sock file.

Description
A way to hide a task from --list output. I'm proposing a task option hidden: bool which hides said task from the output of --list

Alternatives considered

Hard coding a convention such as a leading underscore (i.e. _foo:) being hidden. This has more downsides as it's less clear than hidden: bool.

Additional context

We often had out Toast files to people not familiar with the project they need to build, or even familiar with Toast. For the most part we can get by with just using default tasks, however there are some Toast files where we need to run different target tasks. In these cases users know to use --list to see what options are available. However, many of the tasks shouldn't be used as output targets, and are used as glue tasks. Currently we solve this through convention of a leading underscore (_) means ignore/don't use a task target. We'd prefer being able to hide such tasks from the output of --list entirely.

Hi there. I'm just starting to play with this and it's a super interesting project, so thank you for this!!!

I was trying to run the aws cli in a task and would like to copy in my ~/.aws/credentials.

Any reason not to allow input_paths to be absolute? similar to #268

First, let me say that Toast is really well done! I haven't done anything with Docker/containers before, but the Toast README is succinct and clear, and I'm impressed with how you've kept the scope focused. Nice job, and thanks for sharing it with us!

I'm investigating using Toast from a Mac to run tasks on a remote backend via the DOCKER_HOST env var. However, doing so introduces substantial latency beyond what I'd expect from network overhead. It could be that this is entirely in the Docker engine and out of scope for Toast, but I figured it'd be worth raising here in case anyone has a similar use case and possible suggestion for how to improve the situation.

I'm testing by running this minimal toast.yml:

image: ubuntu:20.04@sha256:c01a10535d9ea3e4065dee4abdb288821b2918929f1e1584287a03e731b9176c
command_prefix: set -euo pipefail
tasks:
  hello:
    cache: false
    command: echo "hello world" > my_output
    output_paths:
      - my_output

and all of the numbers below refer to Docker version 20.10.13, build a224086 on an EC2 ARM machine with Toast 0.45.5.
The only difference is whether I invoke toast from a terminal on the EC2 machine or from my Mac. Ping to the EC2 machine is 20--30 ms.

SSH has a 2s login overhead, so it's out of the question. I'm using TCP instead, here's the command I'm using from my Mac:

LOG_LEVEL=debug DOCKER_HOST=tcp://containertron:2375 toast hello 2>&1 | ts "[%.S]"

(The ts adds a microsecond timestamp to each printed log line.)

Here's a comparison between execution from the Mac and from a terminal on the server. Note that the remote execution via DOCKER_HOST takes about 1s longer than a local execution on the EC2 server:

Again, I have no experience with docker, but can field a few guesses:

• TCP connection overhead
• a bunch of 25 ms round-trips between the Docker CLI and the engine
• protocol overhead (if Docker chats something different over TCP than UNIX socket?)
• DNS resolution overhead across Toast's multiple invocations of docker

Do you have any suggestions for things to investigate to improve the latency here? Thanks!

Description
I have a Docker image that doesn't run under the root user, thus causing su calls to fail when they cannot authenticate interactively.

Instructions to reproduce the bug
This Toast file reproduces the issue:

image: proget.hunterwittenborn.com/docker/makedeb/makedeb-alpha:ubuntu-focal 
command_prefix: set -e
tasks:
  install-dependencies:
    command: |
      echo test

Furthermore, running the following command in that image also verified that su is trying to show an interactive prompt:

su -c 'echo test' makedeb

Environment information:

• Toast version: 0.45.3
• OS: Ubuntu 20.04

Additional context
Using a tiny script to catch Docker CLI calls and prevent removing the container Toast generates shows this as the command used:

[
  "/bin/su",
  "-c",
 "set -e\necho test\n",
 "root"
]

Why's it calling su? Couldn't it just use the --user argument from docker container create, and just omit it when the Toast file doesn't specify a user?

I think something like bash -c '{cmd}' (replacing that entire above command) would be better, and then just allow setting a shell option or something similar in the Toast config to change the shell that's used if needed.

Hi,

I stumbled upon this problem while packaging toast, toast produces a 'toast' bin, sadly this collides with the widely spread 'toast' bin of https://www.quut.com/gsm/ which is a dependency of ffmpeg and is thus very common to be already taken.

( see https://linux.die.net/man/1/toast )

On ArchLinux installing ffmpeg is enough for this, on current Ubuntu it gets taken through libgsm-tools.

This is not necessarily a bug, but I found no prior issue about this and wanted to raise this, I'll package this as toastci for now.

Thanks for consideration.

My toast tasks spent a lot of time repeatedly downloading language dependencies into, e.g., ~/.cargo/registry (Rust) and ~/.m2/ (Java / Clojure).
I've run into two problems trying to share these host dependency caches with containers.

Consider a task definition like:

image: ubuntu:22.04
tasks:
  compile-rust:
    dependencies:
      - install-rust
    mount_paths:
      - ~/.cargo/registry:/root/.cargo/registry
    command: |
      cargo build

Problem 1: Containers with mount_paths are not cachable.

Running toast compile-rust will fail with:

[ERROR] Unable to parse file /Users/dev/software/toast/scratch/toast.yml. Reason: Task compile-rust has mount_paths but does not disable caching. To fix this, set cache: false for this task.

I understand the intent here --- the contents of the mount_paths are untracked by toast and so in general caching build output could lead to reproducibility problems.
However, for the specific need of sharing the host's already-downloaded dependencies, it feels a bit different.
We're trusting language tooling (cargo, mvn, whatever) to handle downloading/validating libraries in a sensible way.
Since toast is already fine caching containers that download arbitrary stuff from the Internet (i.e., isn't aiming for bitwise-reproducible dependency tracking), it feels reasonable to ignore some of the "inputs" for the purposes of caching.

Perhaps we should break out this need out as a separate concept, say cache_paths.
Then we'd have:

• input_paths: tracked by toast, for anything where content changes should re-run command (source code, dependency lockfiles, etc.)
• mount_paths: untracked by toast, for transient data used by commands that never "finish" (e.g., database server)
• cache_paths: untracked by toast, for data that a command will read/write and handle its own dependencies for.

Alternatively, we could allow specifying a "don't cache this" option for specific input paths or a "please allow caching" for containers that use mount_paths, but I think it'll be clearer to have an explicitly named concept.

Problem 2: Intended host paths vary.

I pulled a fast one with:

mount_paths:
  - ~/.cargo/registry:/root/.cargo/registry

It actually needs to be a path relative to the toast file (../../.cargo/registry) or an absolute one (/Users/kevin/.cargo/registry).
Unfortunately, both make it difficult to make toast-driven projects portable, since it'll force you to checkout the project to a specific folder or run as a specific user.

The obvious fix in this case is to write a bit of code to expand ~ into the executing user's home directory, though there be more general ways to solve this need (e.g., allowing for interpretation of environment variables in toast config values beyond just the command string).

If you agree that the need is in-scope for toast, I'm happy to submit PRs in the next few days.

Description

It would be great to be able to invalidate the local cache starting from a specific task, and all tasks that depend on it. i.e. I have several build toast files and would like to be able to invalidate from cloning the repository on, but don't really want to invalidate things like installing deps, setup, etc. cloning tasks naturally don't invalidate themselves because the command is the exact same, even if commits have been happening on the remote repo.

Alternatives considered

A YAML key like,

local_cache_valid_if: |
   cmds_that_return 0
   for_cache_is_valid
   or_return_1 if_invalid_cache

Also, the current using --read-local-cache false which invalidates everything but is less than desirable.

Additional context

None

Description
I'm working on project where I need to import the entire content of the folder except for some files that must not exist in the container. I think it might be useful to support something like excluded_input_paths to exclude some folders/files.

Alternatives considered
The only alternatives that come to mind are:

• declare any necessary path in input_paths
• add a task to remove unnecessary paths

Description
I would like to save task logs if the task failed. I run lots of behavioural tests in a toast task. I want to show nice formatted output to the user (stdout) and save lots of technical logs somewhere (a file) and get it when the task fails (or always). Right now toast saves output_paths only if the task was succesful.

Description
While looking into alternative CI solutions that could use a single config to run task locally as well as in the CI pipeline I found toast, great work!

What I do find myself missing to replace the current CI setup for a couple of client projects is support for using a specific image per task. This is possible in GitLab CI and is quite a nice feature. I guess some sort of "artifact" sharing would be nice in this scenario but as this is based on docker I'm thinking that sharing a volume between the tasks would work.

Alternatives considered
The only alternative I can think of is using multiple toastfiles. However, with that approach, running tasks locally wouldn't be as easy. I guess it's a DX consideration.

Would you be open to a PR implementing parallel task execution?

My use case is developing a web app.
The backend and frontend are implemented in different languages with separate live-reloading infrastructure, so it makes sense to keep those in separate containers / dependency chains.

It'd be nice if a single command could run both tasks in parallel (in the example below, toast dev).
Currently one must open separate terminals and manually run two separate toast tasks.
This makes it harder to hand-off to teammates, since I can't give them a single command to start the dev environment.

One could write a little shell script to spawn two toast invocations, but then responsibilities are muddled ("I thought I was supposed to use toast to run commands, so why do we have some shell scripts?")

For my case, it'd be fine to interleave the output.
Alternatively we could also have an option to prefix each line of the output with the task name.

image: ubuntu:22.04@sha256:0f744430d9643a0ec647a4addcac14b1fbb11424be434165c15e2cc7269f70f8
command_prefix: set -euo pipefail
tasks:

  dev:
    description: Start the entire development system
    dependencies:
      - frontend
      - backend

  frontend:
    description: Start the frontend builder / live-reloader
    command: |
      while true; do
        echo 'frontend...'
        sleep 1
      done

  backend:
    description: Start the backend server
    command: |
      while true; do
        echo 'server...'
        sleep 1
      done

Description

toast --list cannot print out the tasks

Instructions to reproduce the bug

• download the binary from the release
• add hello world toast.yml config file
• run toast --list

$ ~/Downloads/toast --list
[INFO] Here are the tasks that have a description and any environment variables they support:
$

Environment information:

• Toast version: [e.g. 0.0.0]: 0.45.0
• OS: [e.g. macOS Big Sur 11.4 (20F71)]: macOS 11.0

Additional context

Directly running task seems OK.

$ ~/Downloads/toast --version
Toast 0.45.0
$

$ ~/Downloads/toast greet
[INFO] Ready to run 1 task: greet.
[INFO] Running task greet…
[ERROR] Interrupted.
$

Is there a way to specify an alternate toast.yml? If so, I didn't see it.

The reason I'm asking is that I have a project with a variety of required tools. One of them for load testing is node.js based and since a toast.yml appears to be limited to a single base image, I was thinking about creating a 2nd toast.yml (loadtesting.yml) that derives from node:8 since that would be easier than installing node + npm from scratch. Also, since load testing is sort of an independent task, I don't think I'd mind having it managed in a separate yml file.

toast -f loadtesting.yml run

Does that make sense?

Description
I would like to add an option to specify the network namespace akin to what docker cli --network does. In some cases, I would like to drop the container into host network namespace in order to access other resources running on the dev machine. For example, I run a testing consul through the docker compose stack with some testing data. The consul has port mapped the the host network namespace correctly, 8500 for example. In toast, I would like to access that port, but I have no way to specify that.

Alternatives considered
I don't think there is a good alternatives for this particular usecase within toast. I can however to package a runtime container myself using toast only as the build step. I can use normal docker build step to copy the binary from toast build steps and execute docker command myself wrapped by a command runner.

Additional context
The change looks to amend the create_container and container_args call to add the --network flag when executing docker cli. We can either allow any network to be passed in from toast and let docker cli return errors in the case that it is incorrect, or specify only dropping to host network as a boolean flag.

If you are open to this feature, I am happy to submit a PR for this.

Description

Add support for podman (Red Hat's drop in replacement for Docker).

Alternatives considered

• Symbolic Links (ln -s /usr/bin/podman /usr/bin/docker): This does not work on machines that have both podman and Docker installed without more cumbersome PATH manipulation which gets tedious and error prone quickly
• Aliases (alias docker=podman): This does not change anything for toast itself since it's not using the user's shell
• Maintain a private fork with our silly patch: Obviously not the direction we'd like to go

Additional context

Podman supports, among other things, daemonless and rootless containers. At work, we're moving to use podman over Docker where possible. However, there are cases where a tool we're using (i.e. Toast) is using Docker under the hood and we have less control over switching to podman in those instances.

Fortunately, in Toast's case at least, recompiling with this trivial test patch allowed us to use podman in all of our toast files (and we have quite a few non-trivial toast files):

index a20f761..00c5256 100644
--- a/src/docker.rs
+++ b/src/docker.rs
@@ -693,7 +693,7 @@ fn run_attach(error: &str, args: &[String], interrupted: &Arc<AtomicBool>) -> Re

 // Construct a Docker `Command` from an array of arguments.
 fn command(args: &[String]) -> Command {
-    let mut command = Command::new("docker");
+    let mut command = Command::new("podman");
     for arg in args {
         command.arg(arg);
     }

This suggests to me that while fully embracing both podman and docker might be a larger ask, the simple case is extremely simple. We would be perfectly happy with something like an ENV var we could set that swaps out docker for podman, even if it's behind a crazy experimental type name:

i.e.

$ EXPERIMENTAL_TOAST_PLATFORM=podman toast
  # .. uses podman instead

In the patch above, a more realistic example would simply be reading the ENV var (defaulting to docker) instead of hard coding "docker".

Other slightly less desirable options would be a config option to set podman as the container platform, or a CLI option. Both options are less desirably than the ENV var which could easily be set in our .env files and picked up automatically.

Putting this change behind an EXPERIMENTAL_TOAST_* type name would also suggest that this feature is by no means complete, fully vetted, or not subject to radical change.

To be clear, I'm not suggesting fully abstracting over different container platforms. Only supporting literal drop-in replacements for Docker (of which only Podman exists?). Fully "supporting" here could just mean optionally exposing additional features via ENV vars or CLI options, but again that's much much easier than fully abstracting over different platforms 😄

Description
If the inputs to a task contain a broken symlink toast will fail with:

[ERROR] Unable to canonicalize path /Users/dan/development/toast_issue_repo/./broken_ymlink. Reason: No such file or directory (os error 2)

Instructions to reproduce the bug

1. git clone https://github.com/jazzdan/toast_broken_symlink_issue.git
2. cd toast_broken_symlink_issue
3. toast

System information:

• OS: macOS Mojave 10.14.5
• Docker version:

Client: Docker Engine - Community
 Version:           18.09.2
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        6247962
 Built:             Sun Feb 10 04:12:39 2019
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.2
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.6
  Git commit:       6247962
  Built:            Sun Feb 10 04:13:06 2019
  OS/Arch:          linux/amd64
  Experimental:     true

• Toast version: 0.24.0

First great job this tool is really useful!

It would be nice to be able to type something like this:
toast list
and have toast show me the tasks I have and the ability to give tasks some helper text that describes what the task does.

Thanks!