Git Product home page Git Product logo

rundeck-rancher-node-plugin's Introduction

Rancher Nodes Plugin for Rundeck

This plugin implements Rundeck nodes for Rancher-managed Docker containers.

Requirements

The containers must have bash installed for the Node Executer to work.

Features

Rancher Node Resource

Collects nodes from a Rancher controller host.

Features:

  • Project can include multiple environments.
  • API keys are not exposed in configuration.
  • Nodes can be created from containers, services, or both.
  • Can limit selected containers to one per service.
  • Can exclude stopped containers.
  • Can exclude global containers.
  • Can exclude system containers.
  • Can apply a fixed set of tags to all selected containers.
  • Can define node attributes from container labels (configured by regex).
  • Can add tags from container labels (configured by regex).
  • Can add node description (e.g., url) via a label like "com.example.description"

Configuration:

  • Node executor has configurable timeout.
  • Authentication tokens for node executor and file copier are in password storage.
  • The path for authentication tokens is specified in the node source configuration.
  • Users will need to add those keys to storage in addition to entering them as password on the configuration page.

Note:

Note that Rancher uses container ID to construct the URL used to make API requests. When services are upgraded, new containers are created so the API urls used in the node operations cannot work anymore. The container does not exist, so the URL will fail. As long nodes are defined by containers, there is probably no way to avoid this.

The result is that file copies and node executions will not work when performed in the same execution context after an upgrade because the node set is defined in that job context. The Rundeck solution to this issue is to do the upgrade then:

  1. Run a "Refresh Nodes" workflow step
  2. Run the node execute or file copy step as a job reference

Note that refreshing nodes, Rundeck changes the nodes in the global context, but not the running job. So it is essential that the execution is in a job reference and not in the same job context.

Rancher Node Executor

Execute jobs on remote Docker containers managed by the Rancher host.

Features:

  • Can select first container in a service so only one needs to run (filter by "seen:1").
  • Reconstructs the STDERR channel that is missing in output from Rancher API.

Rancher File Copier

Copy files to a node. Can be configured to use Rancher CLI if it is installed and available. Otherwise, assembles files from Base64-encoded parts transmitted via Rancher API.

To distribute to all containers in a stack's service, omit the filter for "seen".

Upgrade Service

Upgrades an existing service. Has required inputs:

  • Docker image
  • Start before stopping

Had many optional inputs:

  • New service labels (JSON Object)
  • New environment variables (JSON Object)
  • New secrets (list of strings)

New Stack

Create a new stack. Has two required inputs:

  • Stack Name (string)
  • Environment ID (string)

Environment ID most correspond to an existing Rancher environment. Stack name must not exist in that environment.

Add Service

Adds a service to an existing stack. Required inputs:

  • Environment ID (string)
  • Stack Name (string)
  • Service Name (string)
  • Docker image

Optional inputs:

  • Data volumes
  • OS environment variables
  • Service labels
  • Secrets
  • OS environment variables to remove
  • Service labels to remove
  • Secrets to remove

Manage Service

Activate, deactivate, or restart a service.

Road Map

  • 0.6.6 Make File Copier binary-safe.
  • 0.7.0 Provide container upgrade node step, with ability to set labels and environment variables.
  • 0.7.x Provide ability to remove labels and environment variables via container upgrade.
  • 0.9.x Provide reasonable if not complete test coverage prior to 1.x

Compatibility

This has been tested with Rundeck 3.1.3 and Rancher 1.6.26.

rundeck-rancher-node-plugin's People

Contributors

kdebisschop avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

rundeck-rancher-node-plugin's Issues

timeout settings

Hi,
I read that timeout for job execution could be set, but I did'nt find how.
Is this features available as a job "option" or as a config option ?
Thank you.
Regards

Render file copy status messages in debug context only

Describe the bug
File copy should be silent when not operating in debug mode.

To Reproduce
Steps to reproduce the behavior:

  1. Execute a file copy step in Rundeck
  2. Output will say something like

copying file: '/var/lib/rundeck/var/tmp/dispatch219377277732210710.tmp' to: 'development_site-service-1:/tmp/1593-31423-development_site-service-1-dispatch-script.tmp.sh'

Expected behavior
In the example, the line above is from running a script on a remote site. We would prefer that the output of the script not be made less clear by status messages that are untended for debugging.

Inline scripts failed time to time

Hello kdebisschop, I hope you are doing well in this strange period :-/

I come back to you regarding a problem I have with "Inline script".
I have started a new rundeck instance from scratch to be sure to not get disturbances from other plugins or features, add the plugin and make a simple bash Inline script executing the hostname command.
Rundeck is connected via the plugin to a Rancher instance managing about 200 containers.

The bug is that time to time, the execution of the script is not done with this error:
"chmod: cannot access /tmp/117-28-custeuro_preprod-custeuro-app-com-rsl-task-pre-prod-custeuro-app-com-1-dispatch-script.tmp.sh: No such file or directory"
and the fact is that the temporary file is present on the container:
ls -lrt /tmp/117-28-custeuro_preprod-custeuro-app-com-rsl-task-pre-prod-custeuro-app-com-1-dispatch-script.tmp.sh
-rw-r--r-- 1 root root 42 14 avril 13:58 /tmp/117-28-custeuro_preprod-custeuro-app-com-rsl-task-pre-prod-custeuro-app-com-1-dispatch-script.tmp.sh
So the script has well been copied, but for any reason it has not been found to be executed.
The screenshots display the same job executed twice, one succeed and the other failed.

script_status
script_ok
script_nok

I tried many different settings modifications (Sync cache,cache delay, refresh nodes before execute, use scp or rancher copy, include services, global containers, system containers ...), but the problem is till present.

If I use the generic command menu of the project to launch the 'hostname' command on nodes, I do not get any issue.

Any idea what could cause this behavior ?

Thank you for your help.

Regards,

Ability to stop a service

Hi,
thank you for sharing this plugin, very very useful when you have a Rancher infrastructure to manage.
In certain case we need to stop a service before upgrading an other, is there a way to achieve this ?
Thank you.

When stopped nodes have the same name as running nodes, node set selection is ambiguous

Describe the bug
Since node name is used to index the nodes, nodes with the same name write over each other. (This was initially surfaced by efforts to increase code coverage).

To Reproduce
Steps to reproduce the behavior:

  1. Add a unit test in node resources that collects containers and has a stopped and running node of the same name
  2. Count nodes in he resulting node set
  3. It will be less than the full set of nodes

File copier requires CLI to be intalled

Describe the bug
Documentation says you can run file copy without CLI, but invocation fails

To Reproduce
Steps to reproduce the behavior:

  1. Enable file copier with no path for cli
  2. Copy a file
  3. See error: "Failed: PluginFailed: property not found: framework.rancher-cli-path"

Expected behavior
Copy should succeed.

Additional Notes
By the way, I encounter a little problem with the last version, due to the need to have the rancher_cli. Documentation says that both can be use, cli and api, but it keeps saying that it did not find the framework.rancher-cli path. I'm not a devellopper at all, more a sysadmin, and I had a look on the code to see if I could find a clue, and I found it, at least I found how to workaround:
file: RancherFileCopier.java
I comment out those 3 lignes:
145 //if (searchPath == null) {
146 // searchPath = framework.getProperty(FMWK_RANCHER_CLI_PATH);
147 //}

"log filter" seems broken

Hi,
the functionality "log filter" seems not to work when using it with rancher nodes.

How to reproduce:

  • 01: simple task "echo RUNDECK:DATA:VARIABLE = 1".
  • 02: simple task "echo VAR = ${data.VARIABLE}"
  • Add a "log filter", global or local same behaviour", "type: Key value data", "pattern: ^RUNDECK:DATA:(.+?)\s*=\s*(.+)$", check "Log data" to display the result.
    job

=> Node: Execute locally: The VARIABLE is detected and the value is assigned
execute_local

=> Node: Dispatch to nodes: The VARIABLE is not detected
execute_node

Thank you for your work.
Regards,

Node Executor fails after 15 minutes

After 15 minutes:

Failed dispatching to node environment_stack-service-1: java.lang.ArrayIndexOutOfBoundsException: 1

Same result when connecting to multiple multiple environments, different hosting centers, different haproxy settings, for entirely different backend jobs. Makes no difference if the connection is or is not sending data.

Capture container upgrade logs

Is your feature request related to a problem? Please describe.

Capture logs output from a container when upgrading it ? Currently, I have to found the container, the host and make a docker logs against it and I was wondering if there would be a way to get this through the api ?

Cannot run cmd on container after upgraded it

Hi,
here under a problem I've got with this job. (Rundeck 3.2.3 docker image)
The goal of the job is to upgrade the container and launch a command on it
Job type: Dispatch to node
step 1: Node step => command: hostname
step 2: Node step => Upgrade Node (Start before stopping: whatever)
step 3: Workflow step => Local command: sleep 10
step 4: Node step => command: hostname

Choose your node and launch, every thing is going find till the last step where you've got:
Failed dispatching to node Node_selected: java.lang.IllegalArgumentException: Expected URL scheme 'http' or 'https' but no colon was found

If, instead of making the last step on the node newly upgraded we do it on an other node, every thing is fine.

Thank you.

Wrong output with start/stop service

Hi,
Version plugin: 0.8
With the stop/start service plugin, the output of the job is "Upgraded $NODE", should be "Started/Stopped $NODE".
Regards

framework.rancher-cli path missing

Thank you once for your work.
I encounter a little problem with the last version, due to the need to have the rancher_cli. Documentation says that both can be use, cli and api, but it keeps saying that it did not find the framework.rancher-cli path. I'm not a devellopper at all, more a sysadmin, and I had a look on the code to see if I could find a clue, and I found it, at least I found how to workaround:
file: RancherFileCopier.java
I comment out those 3 lignes:
145 //if (searchPath == null) {
146 // searchPath = framework.getProperty(FMWK_RANCHER_CLI_PATH);
147 //}

Hope this helps.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.