bob-cd / wendy Goto Github PK

Can we maybe build wendy on Bob without having wendy readily compiled at hand? Maybe we can run some curls to bootstrap wendy on bob or even a small clojure(-)script would be way cooler.
There is a graalvm image from oracle, this should be suitable. https://hub.docker.com/r/oracle/graalvm-ce

when running bob on a different server or host one has to specify this as config (or as env-var). that this is already possible via ~/.wendy.edn should be documented.

Decide the pipeline specification format. Most probably TOML (https://github.com/TheElectronWill/Night-Config) or YAML (https://github.com/clj-commons/clj-yaml)

• Add the running of wendy's unit tests to the steps of bob
• When a commit happens here, we can change the CircleCI steps to run this script instead of the usual stuff
• We can then ditch running the build on CircleCI for wendy
  This way Bob is truly bootstrapped upon another CI!

Originally posted by @lispyclouds in #11 (comment)

No one needs the api-spec from cli. I think this can safely be removed.

Possible solutions:

• Create a map of the config and parse it to the argparse4j calls
• Switch to a data driven parser like cli-matic (needs subparsing abilities to be built/contibed to)

With the complete overhaul of Bob and its REST API. wendy is yet to be updated.
Apart from this since the fact that Bob exposes its api.yaml, Wendy's CLI can be fully generated from the OpenAPI spec on the fly resulting in a smarter, simpler codebase.

REST API of Bob: https://bob-cd.github.io/pages/api-reference.html

Is there an option to get more well-formed output? Currently it looks like this:

[{group test, name buildjson, steps [{cmd echo build started.} {needs_resource source-code, cmd gradle test} {cmd gradle shadowJar, produces_artifact {path build/libs/bob-example-1.0-SNAPSHOT-all.jar, store local, name uberjar}}], image gradle:jdk11, resources [{name source-code, params {repo https://github.com/lispyclouds/bob-example, branch main}, type external, provider git}], vars {LOG_LEVEL error}} {group test, name pipelinejson, steps [{needs_resource source, cmd go test} {needs_resource source, cmd sh -c 'GOOS=linux GOARCH=amd64 go build app.go', produces_artifact {path app, store local, name app}}], image golang:alpine, resources [{name source, params {repo https://github.com/lispyclouds/bob-example, branch main}, type external, provider resource-git}]} {group test, name buildsimplejson, steps [{cmd echo build started.}], image gradle:jdk11}]

I would like to have it at least pretty-printed. I cannot pipe it to jet because the output is without quotes. Changing the println in client.clj to pprint solves both problems.

Any thoughts on this?

the cli params to list registered artifact-stores and external-resources are inconsistent:

root@bob-test:~/wendy# ./wendy artifact-store list
usage: wendy artifact-store [-h] COMMANDS ...
wendy: error: invalid choice: 'list' (choose from 'register', 'show', 'delete')

Did you mean:
        register
        show

root@bob-test:~/wendy# ./wendy external-resource show
usage: wendy external-resource [-h] COMMANDS ...
wendy: error: invalid choice: 'show' (choose from 'register', 'list', 'delete')

I think it should be list for returning a list of the registered systems and show for detailed view on a single system.

wendy pipeline list returns always null although there is correct output in bob-logs.

confirm:
wendy pipeline create -c docs/build.toml
wendy pipeline list

should be a list of the registered pipelines

actually null is returned in any case

The config is transformed from the api.yaml every time it is invoked. That means there is an HTTP-request to Bob and a transformation of the yaml to cli-matic-config every time. Using a ~/.wendy.edn for storing the cli-matic conf should be an option.

Question is when will the cached config be renewed?

I created pipeline from docs/pipeline.json and wanted to run it:

$> bb -m wendy.main pipeline-start --group test --name pipelinejson
** ERROR: **
Option error: Missing option: f


NAME:
 wendy pipeline-start - Submits a new pipeline start request given the pipeline group and name

USAGE:
 wendy pipeline-start [command options] [arguments...]

OPTIONS:
       --f f*      Optional metadata map for runners. If provided must include runner/type
       --group S*  The group of the pipeline
       --name S*   The name of the pipeline
   -?, --help

It seems to require the optional metadata map for --f. It should either be not optional or not required. I don't know what to pass here but couldn't get it working.

COMMANDS:
   get-api-spec         Returns the OpenAPI spec
   health-check         Runs health checks for Bob
   pipeline-create      Creates a new pipeline
   pipeline-delete      Deletes a pipeline
   pipeline-start       Starts a pipeline
   pipeline-stop        Stops a pipeline
   pipeline-pause       Pauses a pipeline
   pipeline-unpause     Unpauses a pipeline
   pipeline-logs        Fetches the logs of a pipeline
   pipeline-status      Fetches the status of the pipeline
   pipeline-artifact-fe Fetches the artifact from pipeline of an artifact store
   pipeline-list        List all pipelines filtered by group, name and status. Case sensitive.
   resource-provider-cr Creates a resource provider with its name and attributes
   resource-provider-de Deletes a resource provider by name
   resource-provider-li Lists all registered external resources by name
   artifact-store-creat Creates an artifact store with its name and attributes
   artifact-store-delet Deletes an artifact store by name
   artifact-store-list  Lists all registered artifact stores by name
   query                Runs a query on Crux, intended for point in time analytics
   get-error            Fetches the next available system error

The commands are too long and are cut after 21 chars.

Currently the only way to build Wendy is manual compilation which is flawed for the reasons:

• GraalVM setup ceremony is long and complex
• Causes a loss of motivation in users to get started quickly
• Really long build times

Possible solutions:

• Make the CI build all binaries
• Make a single latest download link possible, then later go for releases following Bob's release cycle