Maybe we should run nix-channel --update before building dependencies with nix to make sure everyone has the latest local nixpkgs:

$ nix-channel --update

maybe a bob nix update cmd which does that?

This will not really solve the reproducibility issue. Maybe find a way to lock nixpkgs version. We have a way to use a certain version for nixpkgs but maybe is not that user friendly?

Following issues might occur while running git push for multiple repositories in one command by bob.

• How to handle if one of the repositories doesn't have a configured remote?
• What happens when one of the repositories needs to run git pull before pushing the changes to the remote?
• A push command in one of the repositories might also trigger "large file" warnings or, more seriously, errors of this form
• Failed to Push Some Refs for one of the repositories

edited

• run push on all repos in a workspace or just on repos with the same branch name? (=> for now push on all repos)

Allow to define a project name for a Bobfile.

• Improves artifact inspection
• prerequisit to remote artifact upload to projects

Currently the local path is taken as unique identifier / project name

Reproduce:

Bob version: 1510cd623943d95b2d83652715cf39ac08fa7eb7

mkdir new-proj
cd new-proj
bob workspace
bob workspace new

Error:

daniel@beutelsend new-proj ➜ bob workspace new   
panic: bob Workspace Already Initialized

goroutine 1 [running]:
github.com/benchkram/errz.Fatal({0x17dd040, 0xc0004df4d0})
	/home/daniel/go/pkg/mod/github.com/benchkram/[email protected]/error.go:19 +0x65
github.com/benchkram/bob/cli.runWorkspaceNew()
	/home/daniel/workspace/benchkram/bob-server/bob/cli/cmd_workspace_new.go:24 +0x4a
github.com/benchkram/bob/cli.glob..func34(0x21c3700?, {0x15a1268?, 0x0?, 0x0?})
	/home/daniel/workspace/benchkram/bob-server/bob/cli/cmd_workspace_new.go:15 +0x17
github.com/spf13/cobra.(*Command).execute(0x21c3700, {0x22114a0, 0x0, 0x0})
	/home/daniel/go/pkg/mod/github.com/spf13/[email protected]/command.go:860 +0x663
github.com/spf13/cobra.(*Command).ExecuteC(0x21c3980)
	/home/daniel/go/pkg/mod/github.com/spf13/[email protected]/command.go:974 +0x3b4
github.com/spf13/cobra.(*Command).Execute(...)
	/home/daniel/go/pkg/mod/github.com/spf13/[email protected]/command.go:902
github.com/benchkram/bob/cli.Execute()
	/home/daniel/workspace/benchkram/bob-server/bob/cli/cli.go:27 +0x4c
main.main()
	/home/daniel/workspace/benchkram/bob-server/bob/main.go:14 +0x4c

Related: Workspace creation when calling bob workspace is silent.

e.g. bob lint is not defined but it's exit code is 0.

Expectation: exit code should be 1.

If we declare a run task with a relative path on a child bobfile, and call the run task from the parent bobfile, the path won't be properly resolved.
Tested with type: binary. type: compose might also be affected.

The tui spawned by bob run allows to restart the whole environment (hard-restart) using ctrl+r. This will trigger a rebuild and a shutdown of the whole environment + rampup. In case of a compose task it could be handy to only restart changed containers aka soft-restart (docker-compose up can do this out of the box).

• Always run the complete build (already implemented)
• Do not run pre&post scripts on a soft-restart
• Tab aware ctrl-r OR More Generic: track the binary checksum on start and restart only on change (docker-compose already does this). How to do this for cmd:? Probably ignore on restart?
• Don't run docker-compose down by default #98

Lots of useful commands here and it also has a tui

https://github.com/charmbracelet/soft-serve

can also Test against this server :)

How to reproduce

Go in a bob project
Run bob build

Result: An empty .bobcache dir is created inside the project.

Todo: Investigate and fix the issue

TBD

When running a local environment with multiple frontends it is tedious to start them by hand in multiple shells. This issue is about adding support for development with hot reload capacities.

Consider

• Stream cmd output correctly to the tui started by bob run. Consider how terminal reset can be handled correctly (e.g. vue's dev server cleans the output after a successful build)
• The cmd run task could probably implement the noop issue #36

Example bob.yaml

run:
  server:
    type: binary
    path: ./build/server
    dependson:
      - build
  ui:
    cmd: npm run serve
    dir: ./ui
    dependson:
      - server

Description

Having a bobfile like:

build:
  build:
    input: ./hugo
    cmd: cd hugo && go build -o ../build/hugo  main.go

we get the following output:

➜  hugo-contrib bob build
Building nix dependencies...
Succeeded building nix dependencies
Running task build with 0 dependencies

build      	running task...
build      	...done

● ● ● ●
Ran 1 tasks in 6.95s
build              	✔       	(6.95s)

Todo
We should not have nix related text if nix is not enabled

Currently, our install page doesn't show the user how to test that the installation was successful.

We should have simple wording like "Now run bob --help to confirm that everything went fine".

Should be a user error.

?:~/tmp/bobconfigtest$ bob build
2022/05/18 22:42:15 Could not find a Bobfile
: Could not find a Bobfile
github.com/benchkram/errz.Recover
	/........./mod/github.com/benchkram/[email protected]/error.go:46
runtime.gopanic
	/usr/local/go/src/runtime/panic.go:838

• add warning
• dependsOn is not taken as a keyword (but dependson is)

We can remove Store interface

Following reasons:

1. Clean() is implemented only on the local, Done only on the remote
2. ctx was added to the interface only for the remote store purpose
3. In the code they are always used separately so no polymorphism is happening.
4. existence of "not implemented" methods on both implementations

1 & 2 show they really are different behaviours so no benefit of having this interface. Adding new functionalities to one implementation forces us to add params to the second one.

• enable nix build on bob run https://github.com/benchkram/bob/blob/main/bob/run.go#L38
• different nixpkg versions https://github.com/benchkram/bob/blob/main/bob/build_nix.go#L47
• Are nix dependencies resolved correctly if use-nix in umbrella Bobfile is false but true in child Bobfile?

Do not call compose down when restarting on a run command. Its enough to just call compose up again.. will figure out by itself which containers changed and restarts them accordingly.

• Allow to force protocol selection e.g. https, git.
• Prompt for passwords for private repos using https.

E.g. git lfs works with https only.

Remove the implicit forwarding of environment vars to the internal shell when use-nix is enabled as it has lead to multiple problems on CI & in combination with the build in shell of various IDE's.

Make the forwarding of env vars explicit.

bob build --env VAR_ONE
bob build --env VAR_ONE --env VAR_TWO
bob build --env VAR_ONE=value --env VAR_TWO

By default bob clean should only clean artifacts & build info related to the current bobfile's scope. (Now it clears from all projects).

Todo

• Add project name to build info
• Change the bob clean behaviour so that it will clean only the artifacts&build info belonging to the current bob file.
• Add flag -g, --global to bob clean to allow cleaning the entirety of the cache.

For permanent run tasks e.g. in vue serve ui it would be nice to have this as a bob run task.

Example:

runs:
    ui:
        cmd: |
            npm run serve

bob should additionally search for a bobfile in parent directories. This would allow bob commands to be run without going back to the project root.

A user should be able to define a run task that does not run a command/binary or set up a compose environment. This can be useful when grouping tasks together to e.g. bring a dev environment up.

Considering the inference of task type (binary / compose) based on the path provided, as well as the ability to run shell scripts from inside run tasks, an example usecase could be the following:

run:
  compose:
    path: docker-compose.yaml
  server:
    path: ./build/server
    dependson:
      - build
      - compose
  ui:
    cmd: npm --prefix ui run serve
  dev:                                    # <--- noop task
    dependson:
      - server
      - ui

A noop task should not show a status tab on the TUI.

bog git add does not autocomplete on TAB, at least it shows the available options. Plain git's behavior is a bit more useful.

Update to latest version

Recreation

I happened to have an existing config.yml file in my home directory totally unrelated to bob. Putting

source <(bob completion)

in my .bashrc resulted in a segfault. However, it is more easily recreated by:

mkdir test-dir
cd test-dir
touch config.yml
bob

Expected: usage is printed
Actual:

❯ bob_0.4.0_linux_amd64
config file invalid: %!w(<nil>)
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x115e5f1]

goroutine 1 [running]:
github.com/benchkram/bob/cli.glob..func20(0x1fad820?, {0x14660db?, 0x0?, 0x0?})
        /home/runner/work/bob/bob/cli/cmd_root.go:88 +0x1f1
github.com/spf13/cobra.(*Command).execute(0x1fad820, {0xc00003a220, 0x0, 0x0})
        /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:840 +0x563
github.com/spf13/cobra.(*Command).ExecuteC(0x1fad820)
        /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:974 +0x3b4
github.com/spf13/cobra.(*Command).Execute(...)
        /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:902
github.com/benchkram/bob/cli.Execute()
        /home/runner/work/bob/bob/cli/cli.go:27 +0x4c
main.main()
        /home/runner/work/bob/bob/main.go:14 +0x4c

Analysis

There are two aspects to this issue.

• First, an error reading a config file fails to set the GlobalConfig causing the segmentation fault
• Second, a nil error is treated as an error, so if a config.yml file exists at all in the current direcotry it will result in the segmentation fault

Introduce a new feature that handles test data in bob.

Test data is a team-shared storage which provides files needed to run tests. It is not git-lfs, files are versioned but it is easier to handle with less overhead. Eg. you can easily jump between test data versions without affecting the rest of your code base.

How to add files: bob.yaml: specifies directories which are testdata, each with a version tag. Eg:

version: 0.0.2
project: localhost:8100/benchkram/bob-server
use-nix: true
nixpkgs: https://github.com/NixOS/nixpkgs/....

import:
  - docgen
  - ui

variables:
  ...

sync:
  videos:
    path: ./videos
    version: big-feature
  invoice_test_data:
    path: ./tests/invoice_samples
    version: 0.0.1

run:
....

Test data must not collide with git, all files have to be mutually exclusive.

Bob remembers a mapping: local test data path -> file hash to speed up synchronization.

Access-control: not all users (api-tokens) in a project are allowed to push. 'read' and 'read/write' will be the two authorization levels to prevent some users from changing the test data on the server.

Future development:

• Test data can be locally encrypted before uploaded. However, the key can be shared via bob.
• Sync before task: Sync is automatically called everytime "bob build" is called and inputs are within the testdata folder (one has to make sure all test data accessed from code is referenced correctly)

Potential CLI:

daniel@beutelsend my_project ➜ bob testdata --help
Sync (binary) test data via a bob-server.

Usage:
  bob testdata [flags]
  bob testdata [command]

Available Commands:
  ls          list files synced by testdata
  ls-remote   list files on server
  push        make server exactly like local
  pull        make local folder exactly like server


Flags:
  -h, --help       help for build

Global Flags:
  -v, --verbosity int   set verbosity level (default 1)

Use "bob testdata [command] --help" for more information about a command.

Possible work activity:

• have some folder with data

• introduce bob

• add the folder as collection to the bob file, do not define a version tag

• push to remote

• pull on some other machine

• change the data in the folder

• accidentally hit push

• get asked if you want to apply changes to the server

• press no

• go to bob file and add a new version tag

• run bob push

• bob creates a new collection, nothing is lost, bob does not have to prompt

• you are looking for a specific version of a file

• you head to bob-server

• select the path

• you see all the version tags which include this file

A GitHub Action that installs bob would be helpful for users that run bob on CI.
Additionally, the action could run bob install after installing bob to install nix deps.

The url in the servers defintion should not include the api part.

Golands' endpoint discovery currently builds the wrong path eg http://localhost:8080/api/api/ping.

If we notice that nix building time is too long, we should find a way to reduce it.

Todo:

Possible approach: After successfull build, cache the paths in a file and fetch them from there when changing the PATH.

Improve both bob build and bob install commands.

Implement bob git pull

https://github.com/nodejs/help/wiki/Installation

Instead of always running an init task (or always running init-once on startup) run init task only when explicitly required by user.

Steps:

• bob run [task]
• build tasks will be build
• run tasks will be started
• NO INIT TASKS WILL BE STARTED BY DEFAULT
• switch to task in TUI and hit ctrl+i
• Init task for this task will be executed

This will also bring the benefit that such an init task can be run multiple times. E.g. one could reset his test data with this init task.

This could be useful for long-running builds, where the user has went on to do something else while waiting for a build and doesn't have context of the current terminal output.
It could also be very annoying, we should consider when to trigger this sound and possibly allow disabling it.

See title
Also we should probably investigate output-piping-related FIXMEs to fix any bugs that may be present.

We are mostly interested in nix's package management capabilities instead of the full build chain.

References for accessing a nix store

• https://discourse.nixos.org/t/documentation-on-how-the-http-s-binary-cache-api-works/10432/3
• https://github.com/edolstra/nix-serve
• https://github.com/fzakaria/nix-http-binary-cache-api-spec
• https://nixos.wiki/wiki/Binary_Cache
• https://github.com/NixOS/nixpkgs
• https://releases.nixos.org/nix/nix-1.11.9/manual/index.html#idm140737316141296
• https://fzakaria.com/2021/08/12/a-nix-binary-cache-specification.html
• https://gist.github.com/jbeda/5c79d2b1434f0018d693 // nar archive & thesis on nix

Once #97 is merged we should add #131 functionality for run tasks as well.

Currently, is not possible to provide a bobfile with a different name than bob.yaml. This is problematic especially in tests where if we have multiple mock files we always need to rename them to bob.yaml making tests a bit unreadeble. Example

Todo

Add an option to initialize bob with a bobfile with name different than bob.yaml.

This blocks the TUI for a too long time

The commander should return without starting any long running tasks. Upstream code can then start the build after the TUI is started.

For zsh completion you currently have to source the generated completion and then associate it with bob using compdef (see):

source <(bob completion -z)
compdef _bob bob

Better include compdef _bob bob in the end of bob completion -z output.

###Scenario
Have a docker container running which is using e.g. port 8080.
Have a run task which starts a container which also would use port 8080.

###Current behavior
Bob will just exit with either no message or a bad stack trace.

###Expected behavior
Proper user error parsed info from docker-compose message.

Steps to reproduce

On a fresh playground with a bobfile containing:

project: customName

1. Run bob clean to make sure all artifacts are cleared. You can validate that artifacts are cleared by running ls ~/.bobcache/artifacts and checking that the directory is empty.
2. Run bob build to create an artifact. Get the artifact id from ls ~/.bobcache/artifacts.
3. Inspect the artifact. Ex. bob inspect artifact --id 83b9c10614d67dbf

Output:

Artifact Info
  targets:
    targets/run
  metadata:
    taskname: build
    inputHash: 83b9c10614d67dbf
    project: /home/andrei/testare
    createdAt: 06 Jun 22 14:01 +0300
    targetType: path

Desired Output:

Artifact Info
  targets:
    targets/run
  metadata:
    taskname: build
    inputHash: 83b9c10614d67dbf
    project: customName
    createdAt: 06 Jun 22 14:01 +0300
    targetType: path

The project should equal customName not /home/andrei/testare. /home/andrei/testare should be set only if project is missing from bobfile.

The current implementation does not allow to disable the nix cache or choose an alternative location for test isolation.

How to Install nix

• https://nixos.org/manual/nix/stable/installation/installing-binary.html
• wget https://releases.nixos.org/nix/nix-2.6.1/nix-2.6.1-x86_64-linux.tar.xz
• The archive contains a install script for single user install on Linux
• The installer creates a folder /nix and adds entries to either of .bash_profile .bash_login .profile .

Install nix through bob in a distinct location on Linux (macOS is different as there is only multi user install available).
Something like p=$HOME/.nix-profile/etc/profile.d/nix.sh needs to be loaded for each shell from each task.

Add a loading indicator or stream building output to the user. When nix builds take a while it's not obvious that something is happening.

Run commands affected by Init & InitOnce should also get their dependencies from nix.

Todo:

• Nix dependecies in Init
• Nix dependencies in InitOnce
• Nix dependencies in a run task