The script only supports keeping stack envvars between stack updates, not changing them. Even worse, new deployments go without envvars. It would be desirable to allow the user to set ennvars at will.

They could be taken from a file like:

MYSQL_ROOT_PASSWORD=password
ALLOWED_HOSTS=*

Envvars set through the script should overwrite existing ones with the same name, and keep the others.

The Portainer API already has a Env field for this purpose:

[
  {
    "name": "MYSQL_ROOT_PASSWORD",
    "value": "password"
  }
]

Since the beginig of the times this project's name has been "Portainer Stack Utils", which describes what it is and what it does. In order to avoid possible future legal issues with Portainer registered trademark it should be renamed to something that does not include "Portainer". Also, it has evolved to include more than just "stacks", so the name should not limit to "Stack Utils". Whatever name we come up with, it should be reflected by the executable's name (currently psu), which should keep short and easy to type.

Proposals are welcome

The Docker image should use the psu script as ENTRYPOINT

Both scripts contain duplicated code. They should be merged into a single script, and deploy/undeploy behavior should be chosen based on command line arguments.

Hi @hoangphamw, @hoffigk, @idetoile, @jaygridley, @jobleadsorder, @joonyoung-lee, @jpahullo, @kanthi, @LolHens, @mcs07, @mcs07, @ntman4real, @null395922, @rickmclean, @somq, @tortuetorche, @uniconstructor, @xinity, @yeckey, all of you Protainer Stack Utils's stargazers.

I apologize for dragging your attention this massive way. I will be short, promised.

First of all, thank you for starring PSU, it's highly motivating. I would like to have a clearer idea of how is it being used out there in order to stay scoped and serve the most people.

It would be very helpful if you leave a comment briefly describing how you use (or would use, or know someone else uses) PSU, and what do you think is its strongest point.

Best regards,
Juan Carlos

As Portainer API is also written in Go and is OSS, its data structures could be reused in this project instead of being duplicated (as it is done so far). It would allow a more powerful usage of Go templates in --format flags.

There should be a way to get the script version just through the script itself, like

./psu -v

There is a previous related comment in #13.

This isue is being worked on in the several-changes branch. Should be closed once that branch is merged.

There should be a way to get help about script usage just through the script itself, like

./psu -h

Problem

At the moment you can't deploy a stack with non existing configs/secrets. Only external=true is possible.

Goal

1. Similar behaviour like docker stack deploy -c stack.yml test --> deployment of a stack with configs/secrets imported from file.

configs:
  server-config:
    file: config.json

2. actions to create a single config/secrets from file, like docker config create server-config ./config.json or docker secret create my_secret ./secret.json
3. actions to show configs/secrets, like docker config ls or docker secret ls
4. actions to delete a single configs/secrets

CLI

psu config create server-config ./config.json
psu secret create my_secret ./secret.json
psu config ls
psu secret ls
psu config delete server-config
psu secret delete my_secret

Possible Applications

• replace docker cli for local deployments to make use of rbac
• CI/CD deployment of stacks

A while ago this project adopted Semantic Versioning (#9) and so far there are two releases out following that specification 0.1.0 and 0.1.1. As new ones are on their way, there is a need for a clear definition of what is considered a "breaking change" and what is not.

From the spec:

Software using Semantic Versioning MUST declare a public API. This API could be declared in the code itself or exist strictly in documentation. However it is done, it SHOULD be precise and comprehensive.

So, what/where is this project's public API to start with?:

• Executable's input/output format?
• Docker image definition?
• Settings?
• Algorithms (and source code in general)?
• Supported platforms?
• External dependencies?
• A mix of the above?

This issue is for addresing those questions. Findings should be added to the Readme.

We are moving to GitLab. It offers integrated CI/CD, issue boards and project pages built on any website generator, not just Jekyll. This issue is for tracking the progress of such migration.

What will be migrated (see https://gitlab.com/help/user/project/import/github.md):

• Repository
• Issues and Merge Requests (with comments)
• Milestones
• Labels
• Release note descriptions

There are previous related comments in #4.

The project should have releases (like v0.1, v1.2, v2-alpha, etc.). This would allow a cleaner development flow and a clearer usage, so everybody knows about major features additions, deprecations and breaking changes. This would be particularly useful for the Docker images.

Semantic versioning should work fine.

Hi, I am on search for easy stack deployment through portainer instead of docker cli and stumbled over this great tool.

It took me a short time to find out how to compile the binary from master, cause the release is outdated. But then it worked like a charm.

I tried to deploy a simple nginx stack with a few traefik labels. I'am using a local traefik reverse proxy with self signed tls certificate for local development. If I deploy the stack without the configs, the stack is ok. With configs included, the deployment failed (verbose and debug set true). I created the configs by hand in portainer and deployment with docker stack deploy -c stack.1.yml test worked like expected.

• error log

psu stack deploy test -c stack.1.yml

2019/08/02 16:12:24 [Using config file: /home/developer/.psu.yaml]
2019/08/02 16:12:24 [Getting all stacks...]
2019/08/02 16:12:24 [Get stacks request
---
Method: GET
URL: https://docker.local.com/api/stacks?filters={}
Body:

---]
2019/08/02 16:12:24 [Get stacks response
---
Status: 200 OK
Body:
[{"Id":3,"Name":"test","Type":1,"EndpointId":1,"SwarmId":"tk8alsobbjuc147jot4p84gj8","EntryPoint":"docker-compose.yml","Env":[{"name":"domain","value":"test.local.com"}],"ProjectPath":"/data/compose/3","ResourceControl":{"Id":0,"ResourceId":"","SubResourceIds":null,"Type":0,"UserAccesses":null,"TeamAccesses":null,"Public":false}}]

---]
2019/08/02 16:12:24 [Getting stack test...]
2019/08/02 16:12:24 [Stack test found. Updating...]
2019/08/02 16:12:24 [Update stack request
---
Method: PUT
URL: https://docker.local.com/api/stacks/3?endpointId=1
Body:
{"StackFileContent":"version: '3.7'\n\nservices:\n  server:\n    image: nginx:alpine\n    deploy:\n      labels:\n        \"traefik.enable\": \"true\"\n        \"traefik.port\": \"80\"\n        \"traefik.docker.network\": \"web\"\n        \"traefik.frontend.rule\": \"Host:${domain:-test.local.com}\"\n        \"traefik.backend.loadbalancer.method\": \"wrr\"\n        \"traefik.frontend.headers.SSLRedirect\": \"true\"\n    configs:\n      - source: index.html\n        target: /usr/share/nginx/html/index.html\n    networks:\n      external:\n\nnetworks:\n  external:\n    external: true\n    name: web\n\nconfigs:\n  index.html:\n    external: false\n    name: index.html","Env":[{"name":"domain","value":"test.local..com"}],"Prune":false}
---]
2019/08/02 16:12:24 [Update stack response
---
Status: 500 Internal Server Error
Body:
{"err":"read /data/compose/3: is a directory\n","details":"read /data/compose/3: is a directory\n"}

---]
2019/08/02 16:12:24 Error: read /data/compose/3: is a directory
: read /data/compose/3: is a directory

• stack.1.yml

version: '3.7'

services:
  server:
    image: nginx:alpine
    deploy:
      labels:
        "traefik.enable": "true"
        "traefik.port": "80"
        "traefik.docker.network": "web"
        "traefik.frontend.rule": "Host:${domain:-test.local.com}"
        "traefik.backend.loadbalancer.method": "wrr"
        "traefik.frontend.headers.SSLRedirect": "true"
    configs:
      - source: index.html
        target: /usr/share/nginx/html/index.html
    networks:
      external:

networks:
  external:
    external: true
    name: web

configs:
  index.html:
    external: false
    name: index.html

• index.hml

<!DOCTYPE html>
<html>
<body>

<h1>My First Heading</h1>
<p>My first paragraph.</p>

</body>
</html> 

The project currently follows GitHub Flow and it's git branching model. It's been a good choice so far, but there are releases now and it's necessary to maintain a first 0.x version while also working in the upcoming 1.x version. I'm thinking on GitLab Flow.

Documentation should include how to use the Docker image

Is your feature request related to a problem? Please describe.
The tool falls short in managing configs, secrets, containers, images, etc. It's not meant to be a clone of the Docker cli, but still those features are missed.

Describe the solution you'd like
Instead of reinventing the wheel, allow the tool to be used as a proxy to an endpoint's Docker API at /api/endpoint/{id}/docker, so it can be managed with a local Docker client. Something like:

psu proxy --url http://portainer.local --user admin --password a --endpoint primary --address 127.0.0.1:11000

The, in another shell:

DOCKER_HOST=127.0.0.1:11000
docker config create server-config ./config.json
docker secret create my_secret ./secret.json
docker config ls
docker secret ls
docker config delete server-config
docker secret delete my_secret
...

Describe alternatives you've considered
See #24

Flags should have a complete and a short form, something like --action and -a. This would help using and particularly reading them. A command like this one:

./psu -a deploy -u admin -p password -l http://portainer.local -n mystack -c /path/to/docker-compose.yml -g /path/to/env_vars_file

could also look like this one:

./psu --action deploy --user admin --password password --instance http://portainer.local --stack mystack --compose-file /path/to/docker-compose.yml --env-file /path/to/env_vars_file

The scripts should provide better error messages.

Ideas

• Adding the --check-status flag to httpie calls would allow to check their HTTP status code.
• Parsing JSON responses and checking for "err" and "details" attributes would allow to reuse error mesages from Portainer API.

SSL verification should be determined by an ENV var (like, user, password, etc.).

Hey !

I have an issue when I try to deploy:

./psu: line 290: bad substitution: no closing `}' in "${docker_compose_file_content//"/'\"'}"

I share my docker-compose.yml file:

version: '2'

networks:
  default:

services:

  api:
    image: ...
    ports:
      - '3000:3000'
    environment:
      - NODE_ENV=production
    networks:
      default:
        aliases:
          - api

  front:
    image: ...
    ports:
      - '80:80'
    environment:
      - NODE_ENV=production
    networks:
      default:
        aliases:
          - front

Do you have any solution for that ?
Thank for your help :)

Describe the bug
When using PSU as a proxy to an endpoint's Docker daemon, commands requiring a websocket connection (like docker attach, docker exec, docker system events) fail with an unable to upgrade to tcp, received 200 error or just hang up.

To Reproduce
Given:

• There is a "primary" endpoint
• There is a "backend" container running on that endpoint
• There is a Docker client installed locally

Steps to reproduce the behavior:

1. Run psu proxy --endpoint primary --port 12345
2. In another shell, run export DOCKER_HOST=tcp://localhost:12345
3. Run docker exec backend echo "hello"
4. See error

Expected behavior
PSU should correctly proxy the request by handling the protocol upgrade

Standard/Error output

unable to upgrade to tcp, received 200

Standalone executable binary (please complete the following information):

• OS: Linux
• Shell interpreter: Bash
• Portainer version: 1.21.0
• Program version: 2.0.0-alpha.3

The script should be tested on each push in an easy, reproducible and auditable (public) way. A bonus benefit would be the ability to run tests against several Portainer versions to ensure compatibility.

Action selection, which is currently handled with a flag, should be an argument instead, like this:

psu deploy --user admin --password password --instance http://portainer.local --stack mystack --compose-file /path/to/docker-compose.yml --env-file /path/to/env_vars_file