Git Product home page Git Product logo

installer's Introduction

CONSUL DEMOCRACY Installer Build status on Ubuntu

CONSUL DEMOCRACY installer for production environments

Using Ansible, it will install and configure the following:

  • Ruby
  • Rails
  • Postgres
  • Nginx
  • Puma
  • SMTP
  • Memcached
  • DelayedJobs
  • HTTPS
  • Capistrano

It will also create a deploy user to install these libraries

Screencast

How to setup CONSUL DEMOCRACY for a production environment

Prerequisities

A remote server with one of the supported distributions:

  • Ubuntu 20.04 x64
  • Ubuntu 22.04 x64
  • Ubuntu 24.04 x64
  • Debian Bullseye x64
  • Debian Bookworm x64

Access to a remote server via public ssh key without password. The default user is deploy but you can use any user with sudo privileges.

ssh root@remote-server-ip-address

Updated system package versions

sudo apt-get update

Python 3 installed in the remote server

Running the installer

The following commands must be executed in your local machine

Install Ansible >= 2.7

Get the Ansible Playbook

git clone https://github.com/consuldemocracy/installer
cd installer

Create your local hosts file

cp hosts.example hosts

Update your local hosts file with the remote server's ip address

remote-server-ip-address (maintain other default options)

Run the ansible playbook

ansible-playbook -v consul.yml -i hosts

Note about old versions: if you've already used the installer before version 1.1 was released, you might need to remove your ~/.ansible folder.

Visit remote-server-ip-address in your browser and you should see CONSUL DEMOCRACY running!

Admin user

You can sign in to the application with the default admin user:

Deploys with Capistrano

To restart the server and deploy new code to the server we have to configure Capistrano.

Screencast

How to setup Capistrano

Create your fork

Setup locally for your development environment

Checkout the latest stable version:

git checkout origin/2.2.0 -b stable

Create your deploy-secrets.yml

cp config/deploy-secrets.yml.example config/deploy-secrets.yml

Update deploy-secrets.yml with your server's info

production:
  deploy_to: "/home/deploy/consul"
  ssh_port: "22"
  server1: "your_remote_ip_address"
  user: "deploy"

Update your repo_url in deploy.rb

set :repo_url, 'https://github.com/your_github_username/consuldemocracy.git'

Make a change in a view and push it your fork in Github

git add .
git commit -a -m "Add sample text to homepage"
git push origin stable

Deploy to production

branch=stable cap production deploy

You should now see that change at your remote server's ip address

Email configuration

Screencast

How to setup email deliveries

Screencast update: The Installer now configures a queue to send emails asynchronously. Thus you will not see a 500 error when there is a misconfiguration, as the email is sent asyncronously and the error will be raised in the queue. To see email error logs open the rails console (cd /home/deploy/consul/current/ && bin/rails c -e production) and search for the last error in the queue Delayed::Job.last.last_error)

Update the following file in your production server: /home/deploy/consul/shared/config/secrets.yml

You want to change this block of code for your production environment and use your own SMTP credentials:

  mailer_delivery_method: "smtp"
  smtp_settings:
    :address:              "smtp.example.com"
    :port:                 "25"
    :domain:               "your_domain.com"
    :user_name:            "username"
    :password:             "password"
    :authentication:       "plain"
    :enable_starttls_auto: true

And restart the server running this command from your local CONSUL DEMOCRACY installation (see Deploys with Capistrano for details).

cap production deploy:restart

Once you setup your domain, depending on your SMTP provider, you will have to do two things:

  • Update the server_name with your domain in /home/deploy/consul/shared/config/secrets.yml.
  • Update the sender_email_address from the admin section (remote-server-ip-address/admin/settings)

If your SMTP provider uses an authentication other than plain, check out the Rails docs on email configuration for the different authentation options.

Staging server

To setup a staging server to try things out before deploying to a production server:

Update your local hosts file with the staging server's ip address

remote-server-ip-address (maintain other default options)

And run the playbook with an extra var "env":

ansible-playbook -v consul.yml --extra-vars "env=staging" -i hosts

Visit remote-server-ip-address in your browser and you should now see CONSUL DEMOCRACY running in your staging server.

SSL with LetsEncrypt

Using https instead of http is an important security configuration. Before you begin, you will need to either buy a domain or get access to the configuration of an existing domain. Next, you need to make sure you have an A Record in the DNS configuration of your domain, pointing to the correponding IP address of your server. You can check if your domain is correctly configured at this url https://dnschecker.org/, where you should see your IP address when searching for your domain name.

Once you have that setup we need to configure the Installer to use your domain in the application.

First, uncomment the domain variable in the configuration file and update it with your domain name:

#domain: "your_domain.com"

Next, uncomment the letsencrypt_email variable in the configuration file and update it with a valid email address:

#letsencrypt_email: "[email protected]"

Re-run the installer:

ansible-playbook -v consul.yml -i hosts

You should now be able to see the application running at https://your_domain.com in your browser.

Configuration Variables

These are the main configuration variables:

# Server Timezone
timezone: Europe/Madrid

# Authorized Hosts
ssh_public_key_path: "~/.ssh/id_rsa.pub"
ansible_ssh_private_key_file: "~/.ssh/id_rsa"

#Postgresql
database_name: "consul_production"
database_user: "deploy"
database_password: "change_me"
database_hostname: "localhost"

#SMTP
smtp_address:        "smtp.example.com"
smtp_port:           25
smtp_domain:         "your_domain.com"
smtp_user_name:      "username"
smtp_password:       "password"
smtp_authentication: "plain"

If you are on Ubuntu and would like to use its default sudo group instead of wheel, change the deploy_group variable to:

deploy_group: sudo

There are many more variables available check them out here

Other deployment options

Split database from application code

The consul playbook creates the database on the same server as the application code. If you are using a cloud host that offers managed databases (such as AWS RDS, Azure Databases, or Google Cloud SQL), we recommend using that instead.

To set up the application by itself:

  1. Fork this repository.
  2. Specify your database credentials (see the database_* group variables) in a vault.
  3. Run the app playbook instead of the consul one against a clean server.
ansible-playbook -v app.yml -i hosts

Platform-as-a-Service (PaaS)

Aside from just using managed databases, you might also look into platform-as-a-service options (like Azure App Service or Google App Engine) to not have to manage a server at all.

No root access

By default the installer assumes you can log in as root. The root user will only be used once to login and create a deploy user. The deploy user is the one that will actually install all libraries and is the user that must be used to login to the server to do maintenance tasks.

If you do not have root access, you will need your system administrator to grant you sudo privileges for a deploy user in the wheel group without password. You will also need to change the variable ansible_user to deploy in your hosts file.

Using a different user than deploy

Change the variable deploy_user to the username you would like to use.

Ansible Documentation

http://docs.ansible.com/

Roadmap

Cross platform compatibility (Ubuntu, CentOS)

Greater diversity of interchangeable roles (nginx/apache, unicorn/puma/passenger, rvm/rbenv)

How to contribute

Support

Join the chat at https://gitter.im/consul/consul

License

Code published under AFFERO GPL v3 (see LICENSE-AGPLv3.txt)

installer's People

Contributors

afeld avatar allthesignals avatar bkmgit avatar catileptic avatar cronopioelectronico avatar dependabot[bot] avatar javierm avatar javimoral avatar mariacheca avatar microweb10 avatar pierremesure avatar pomerange avatar senen avatar sikado avatar taitus avatar voodoorai2000 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

installer's Issues

Task Update rvm returs fatal

Hello, Im trying to deploy Consul on a vps.
While trying to run the playbook I get the following Error at the update rvm task:

TASK [galaxy/rvm_io.ruby : Update rvm] *****************************************
fatal: [149.210.191.114]: FAILED! => {"changed": false, "cmd": "/home/deploy/.rvm/bin/rvm get stable && /home/deploy/.rvm/bin/rvm reload", "delta": "0:00:00.233758", "end": "2018-10-16 13:08:11.563976", "msg": "non-zero return code", "rc": 1, "start": "2018-10-16 13:08:11.330218", "stderr": "Archives path '/usr/share/rvm/archives' not writable, aborting.", "stderr_lines": ["Archives path '/usr/share/rvm/archives' not writable, aborting."], "stdout": "", "stdout_lines": []}

My local machine runs Ubuntu 18.04.1 LTS
And my remote server runs: Ubuntu 16.04.5 LTS

I do not have direct Acces to the root of the remote server but I have Acces to an user with the same rights. So I adjusted the host file and the playbook to look like this:

Host:

[servers]
xxx.xxx.xxx.xxx ansible_ssh_extra_args='-o StrictHostKeyChecking=no' ansible_ssh_private_key_file='~/.ssh/id_rsa' ansible_port=22 ansible_ssh_user='consuladmin'

Playbook:

---

- hosts: all
  remote_user: consuladmin
  become: yes
  vars_files:
    - ./vars.yml
  roles:
    - postgresql

Any help?

'Start Unicorn' task fails due to syntax error in production.rb

Running installer via Ansible for a production environment on a fresh build of Ubuntu 16.04 LTS

Output:

TASK [unicorn : Start Unicorn] *******************************************************************************************************
fatal: [178.79.181.38]: FAILED! => {"changed": true, "cmd": "/home/deploy/.rvm/gems/ruby-2.3.2/wrappers/unicorn -c config/unicorn.rb -E production -D", "delta": "0:00:04.677967", "end": "2019-05-03 16:57:27.613531", "msg": "non-zero return code", "rc": 1, "start": "2019-05-03 16:57:22.935564", "stderr": "master failed to start, check stderr log for details", "stderr_lines": ["master failed to start, check stderr log for details"], "stdout": "", "stdout_lines": []}
        to retry, use: --limit @/root/installer/consul.retry

Error log (unicorn.stderr.log):

I, [2019-05-03T16:57:23.671088 #10526]  INFO -- : Refreshing Gem list
	/home/deploy/.rvm/gems/ruby-2.3.2/gems/activesupport-5.0.7.2/lib/active_support/dependencies.rb:293:in `require': /home/deploy/consul/config/environments/production.rb:104: syntax error, unexpected ':', expecting keyword_end (SyntaxError)
	    port:                 "25",
	         ^
	/home/deploy/consul/config/environments/production.rb:104: syntax error, unexpected ',', expecting keyword_end
	/home/deploy/consul/config/environments/production.rb:109: syntax error, unexpected end-of-input, expecting keyword_end

production.rb:

  if ENV["RAILS_LOG_TO_STDOUT"].present?
    logger           = ActiveSupport::Logger.new(STDOUT)
    logger.formatter = config.log_formatter
    port:                 "25", <--- THIS IS LINE 104
  end

Error when installing Postgres extensions

I just tried to install Consul with Ansible on a Ubuntu 16.04 LTS machine and I got the following error message:

TASK [postgresql : Add PostgreSQL extensions] *********************************************************************************************************************************************************************
ok: [95.216.150.39] => (item=plpgsql) => changed=false 
  ansible_loop_var: item
  db: consul_production
  ext: plpgsql
  item: plpgsql
  queries: []
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: OperationalError: could not open extension control file "/usr/share/postgresql/9.5/extension/unaccent.control": No such file or directory
failed: [95.216.150.39] (item=unaccent) => changed=false 
  ansible_loop_var: item
  item: unaccent
  msg: |-
    Database query failed: could not open extension control file "/usr/share/postgresql/9.5/extension/unaccent.control": No such file or directory
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: OperationalError: could not open extension control file "/usr/share/postgresql/9.5/extension/pg_trgm.control": No such file or directory
failed: [95.216.150.39] (item=pg_trgm) => changed=false 
  ansible_loop_var: item
  item: pg_trgm
  msg: |-
    Database query failed: could not open extension control file "/usr/share/postgresql/9.5/extension/pg_trgm.control": No such file or directory

I solved it by manually installing postgresql-contrib:

sudo apt install postgresql-contrib

I didn't have any problem before so this is maybe due to the cleaning of #76. In any case, I suggest we add postgresql-contrib to the list of dependencies to avoid this error.

Context

This crash happened on a VPS on Ubuntu 16.04 hosted on Hetzner.

ERROR: installer/roles/postgresql/tasks/main.yml': line 2, column 3

Hi,

The installer stucks at ansible-playbook execution as follows inside a brand new Ubuntu 16.04VM:

$ sudo -S ansible-playbook consul.yml -i hosts --extra-vars "target=servers"         
ERROR! no action detected in task

The error appears to have been in '/home/zenny/consul_installer/roles/postgresql/tasks/main.yml': line 2, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

---
- include_role:
  ^ here

Add imagemagick

What

To upload images the required library should be installed: imagemagick

Add Unicorn init script

With the clean install there is not a unicorn init script, I think it is necessary for start Consul after reboot or to restart it easy.

How

This is an standar file to create a unicorn init script that could be used to make a template for Ansible and a task to deploy it.


### BEGIN INIT INFO
# Provides:          unicorn
# Required-Start:    $all
# Required-Stop:     $all
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: starts the unicorn app server
# Description:       starts unicorn using start-stop-daemon
### END INIT INFO

set -e

USAGE="Usage: $0 <start|stop|restart|upgrade|rotate|force-stop>"

# app settings
USER="deploy"
APP_NAME="appname"
APP_ROOT="/home/$USER/$APP_NAME"
ENV="production"

# environment settings
PATH="/home/$USER/.rbenv/shims:/home/$USER/.rbenv/bin:$PATH"
CMD="cd $APP_ROOT && bundle exec unicorn -c config/unicorn.rb -E $ENV -D"
PID="$APP_ROOT/shared/pids/unicorn.pid"
OLD_PID="$PID.oldbin"

# make sure the app exists
cd $APP_ROOT || exit 1

sig () {
  test -s "$PID" && kill -$1 `cat $PID`
}

oldsig () {
  test -s $OLD_PID && kill -$1 `cat $OLD_PID`
}

case $1 in
  start)
    sig 0 && echo >&2 "Already running" && exit 0
    echo "Starting $APP_NAME"
    su - $USER -c "$CMD"
    ;;
  stop)
    echo "Stopping $APP_NAME"
    sig QUIT && exit 0
    echo >&2 "Not running"
    ;;
  force-stop)
    echo "Force stopping $APP_NAME"
    sig TERM && exit 0
    echo >&2 "Not running"
    ;;
  restart|reload|upgrade)
    sig USR2 && echo "reloaded $APP_NAME" && exit 0
    echo >&2 "Couldn't reload, starting '$CMD' instead"
    $CMD
    ;;
  rotate)
    sig USR1 && echo rotated logs OK && exit 0
    echo >&2 "Couldn't rotate logs" && exit 1
    ;;
  *)
    echo >&2 $USAGE
    exit 1
    ;;
esac```

Unicorn can no longer be exetuted as deploy user

Hello,
today I deployed Consul on a new server. The installation worked straightforward, except that in my previous attempt. about a month ago, I was able to schedule unicorn to start at boot using user "deploy", in this new instance it can only be run as root.

If I "sudo chown -R deploy:wheel /home/deploy/consul" then I'm able to start it againg with user "deploy".

What's best?

Thank you for your help
Best regards

Support other GNU/Linux distributions

It would be great if we could make the installer fully compatible with other distributions, like Debian, instead of supporting just Ubuntu.

Note: in order to support other distributions we'd have to find a way to run the tests on machines with those distributions.

Add server name to secrets

Context

Emails send by the application sometimes have links, these links use the server_name to generate the corresponding urls

What

Add server_name to the secrets.yml

Why

Links in emails do not have the correct urls

How

Use the IP by the default and add instructions to configure a variable with the real domain name in group_vars/all

Remove duplicate file structure

Context

The installer installs CONSUL in 2 folders:
/home/deploy/consul
/home/deploy/consul/current

This is due to an previous version of the installer that installed it in /home/deploy/consul
And an updated version of the installer that uses capistrano and installs it at /home/deploy/consul/current

What

Make the installer install a single copy of CONSUL at /home/deploy/consul/current

Fix user permissions when copying deploy.rb file locally

Context

Some users are having permission problems when running the installer.

TASK [capistrano : Copy deploy.rb (locally)] ***********************************
fatal: [*********]: FAILED! => {"changed": false, "checksum": "******", 
"dest": "/home/consul/installer/consul/config/deploy.rb", "gid": 0, 
"group": "root", "mode": "0644", "msg": "chown failed: failed to look up user deploy",
 "owner": "root", "path": "/home/consul/installer/consul/config/deploy.rb", 
"size": 1922, "state": "file", "uid": 0}
    to retry, use: --limit @/home/consul/installer/consul.retry

PLAY RECAP *********************************************************************
192.168.1.10               : ok=47   changed=3    unreachable=0    failed=1

How to fix

An option would be to allow running the installer directly from the host machine to avoid these problems. Another option would be to create the necessary user groups in the local machine.

Proposal wont go over HTTPS

Hey there,

I have Consul deployed on an Ubuntu server. I made the site secure with SSL with the use of certbot (Let's Encrypt certificate) The whole site is encrypted except for any of the proposal pages.
To be exact the following pages are not secure: /proposals/new, /legislation/processes/../proposals/new and /budgets/../investments

If I set [force_ssl] https://github.com/consul/consul/blob/master/config/environments/production.rb#L45) to true ill recieve an ERR_TOO_MANY_REDIRECTS error in browser.

these are my nginx conf files:
/etc/nginx/sites-enabled/default

upstream app {
        server unix:/home/deploy/consul/sockets/unicorn.sock;
}

server {
        listen 80 default_server;
        listen [::]:80 default_server;

        root /home/deploy/consul/public;

        server_name _;

        try_files $uri/index.html $uri @app;
        location @app {
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $http_host;
                proxy_redirect off;
                proxy_pass http://app;
        }
}

server {

        root /home/deploy/consul/public;
    server_name mydomain.com; # managed by Certbot

        try_files $uri/index.html $uri @app;
        location @app {
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $http_host;
                proxy_redirect off;
                proxy_pass http://app;
        }

    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/mydomain.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/mydomain.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

server {
    if ($host = consulemmen.ddns.net) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

        listen 80 ;
        listen [::]:80 ;
    server_name consulemmen.ddns.net;
    return 404; # managed by Certbot
}

/etc/nginx/sites-available/default

server {
        listen 80 default_server;
        listen [::]:80 default_server;

        root /var/www/html;

        # Add index.php to the list if you are using PHP
        index index.html index.htm index.nginx-debian.html;

        server_name _;

        location / {
                # First attempt to serve request as file, then
                # as directory, then fall back to displaying a 404.
                try_files $uri $uri/ =404;
        }
}

Consolidate documentation

Right now, most of the documentation for production setup is in the README of this repository, while a bit lives in the documentation site. I suggest deciding on one place (for now - can change later), or at least linking between them.

Update Documentation regarding initial ssh connection

What

In order to make the initial connection to the remote server, the playbook expects access by the root user, using a public key

We should update the documentation to reflect that having the public_key in the server is a prerequisite to running the playbook

Configure consul repo

Right now this installer clones consul/consul repo but its recommended to use a fork in order to be able to make customizations and store them in version control.

So I guess we could add steps to:

  1. Create a consul fork (if it doesn't exist already)
    https://consul_docs.gitbooks.io/docs/content/en/forks/create.html
  2. Configure the fork's url at https://github.com/consul/installer/blob/configuration/roles/rails/tasks/main.yml#L10 and

For the second step, it would be nice to avoid changing the installer source code. I guess passing the git repo url as a parameter of the sudo ansible-playbook -v consul.yml -i hosts --extra-vars "target=vagrant" could work? But the best solution would be to store a particular installation details on a file.. maybe consul.yml? I don't know πŸ™ˆ

TASK [Gathering Facts] fatal: UNREACHABLE!

Hi,

Trying to run directly into the vanilla Ubuntu 14.06 instance (LXC), but encountered the following error:

$ sudo -S ansible-playbook consul.yml -i hosts --extra-vars "target=servers"

PLAY [Create deploy user] *******************************************************************************************

TASK [Gathering Facts] **********************************************************************************************
fatal: [192.168.10.115]: UNREACHABLE! => {"changed": false, "msg": "Authentication failure.", "unreachable": true}
        to retry, use: --limit @/home/zenny/consul_installer/consul.retry

PLAY RECAP **********************************************************************************************************
192.168.10.115             : ok=0    changed=0    unreachable=1    failed=0

Add to README how to install Ansible on Ubuntu

Why

Right now we only have documentation on how to install Ansible with brew, which is popular for mac users, but not very common for Linux users

What

Add documentation on how to setup Ansible for Linux, particularly Ubuntu to start with

How

Check out the @zenny's walkthrough as a reference πŸ‘Œ

Update ruby version to 2.3.3

What

COSNUL uses 2.3.3 but the playbook runs Ruby version 2.3.2
Make the playbook consistent with the current CONSUL version

Ansible & Ubuntu 18.04 incompatibilities

Original issue

We've found to incompatibilities between Ansible configuration and Ubuntu 18.04.

Default version of PostgreSQL in Ubuntu 18.04 is version 10, which raises the following error:

"UNENCRYPTED option is no longer supported in PG10"

This error has an easy solution, telling Ansible to encrypt user passwords in database

The other error raises when trying to install libcurl3, due to incompatibilities. Installing libcurl4 instead has worked fine so far for us.

Comments by @valdis

Error 1:

TASK [system : Add Node repository] ***************************************************************************************************
fatal: [157.230.119.69]: FAILED! => changed=false
  msg: 'W:GPG error: https://deb.nodesource.com/node_6.x bionic InRelease: The following signatures couldn''t be verified because the public key is not available: NO_PUBKEY 1655A0AB68576280, E:The repository ''https://deb.nodesource.com/node_6.x bionic InRelease'' is not signed.'

Manual solution:

root@ubuntu18-server:~# curl --silent https://deb.nodesource.com/gpgkey/nodesource.gpg.key | sudo apt-key add -

Error 2:

TASK [system : Install Tools] *********************************************************************************************************
fatal: [157.230.119.69]: FAILED! => changed=false
  cache_update_time: 1572695546
  cache_updated: true
  msg: |-
    '/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold"      install 'build-essential' 'zlib1g-dev' 'libssl-dev' 'libreadline-dev' 'libyaml-dev' 'libxml2-dev' 'libxslt1-dev' 'libffi-dev' 'libcurl3' 'libcurl4-openssl-dev' 'nodejs' 'libpq-dev' 'imagemagick' 'ruby-dev'' failed: E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.
  rc: 100
  stderr: |-
    E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.
  stderr_lines: <omitted>
  stdout: |-
    Reading package lists...
    Building dependency tree...
    Reading state information...
    Some packages could not be installed. This may mean that you have
    requested an impossible situation or if you are using the unstable
    distribution that some required packages have not yet been created
    or been moved out of Incoming.
    The following information may help to resolve the situation:

    The following packages have unmet dependencies:
     libcurl3 : Conflicts: libcurl4 but 7.58.0-2ubuntu3.8 is to be installed
     libcurl4 : Conflicts: libcurl3 but 7.58.0-2ubuntu2 is to be installed
  stdout_lines: <omitted>

Manual solution:

root@ubuntu18-server:~# apt-get remove libcurl4
root@ubuntu18-server:~# apt-get install libcurl3

Error 3:

TASK [postgresql : Create PostgreSQL database] ****************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ModuleNotFoundError: No module named 'psycopg2'
fatal: [157.230.119.69]: FAILED! => changed=false
  msg: Failed to import the required Python library (psycopg2) on consul's Python /usr/bin/python3. Please read module documentation and install in the appropriate location

Manual solution:

root@ubuntu18-server:~# apt install python3-psycopg2

Error 4:

TASK [galaxy/rvm.ruby : Install rvm] **************************************************************************************************
fatal: [157.230.119.69]: FAILED! => changed=true
  cmd: |-
    /tmp/rvm-installer.sh stable --path ~/.rvm --auto-dotfiles --user-install
  delta: '0:00:00.046766'
  end: '2019-11-02 12:15:01.972168'
  msg: non-zero return code
  rc: 4
  start: '2019-11-02 12:15:01.925402'
  stderr: |-
    WARN: ...the preceeding error with code 200 occurred while fetching https://api.github.com/repos/rvm/rvm/tags
    WARN: ...the preceeding error with code 200 occurred while fetching https://api.bitbucket.org/2.0/repositories/mpapis/rvm/refs/tags?sort=-name&pagelen=20

    ERROR: Exhausted all sources trying to fetch version 'latest' of RVM!
  stderr_lines: <omitted>
  stdout: |-
    Turning on auto dotfiles mode.
    Turning on user install mode.
  stdout_lines: <omitted>

Manual solution:

root@ubuntu18-server:~# gpg --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB

Add documentation on how to setup without root access

Context

The installer assumes we can login as the root user using a public ssh key and without password. Which is only used to create a deploy user, who is the one that will actually install all libraries.

However some developers are not granted root user access and cannot run the installer with the default configuration.

Objectives

Allow the installer to be run with any user with sudo privileges.

How

After doing a couple of tests, it seems we only need to add this line: remote_user: name_of_user here.

Installer - Spec Fail

I was having a deploy running v0.19. Then I rebased it on top of v1.0.0 and did a Capistrano deployment. Then I noticed some thing like (or at least that is what I have now).

01:36 deploy:restart
      01 /etc/init.d/unicorn_consul restart
      01 bash: /etc/init.d/unicorn_consul: No such file or directory
      01
(Backtrace restricted to imported tasks)
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as [email protected]: /etc/init.d/unicorn_consul restart exit status: 127
/etc/init.d/unicorn_consul restart stdout: bash: /etc/init.d/unicorn_consul: No such file or directory

Then I thought of running the installer as some thing seemed to be missing. After some fixes the installation got stuck in specs. Then I thought of running it on a bare Ubuntu 16.04 server with a not modified (except the base config) and ended up in the same error.

I didn't see any thing related to the error in the issue list so thought of opening an issue.

BTW is there upgrade docs that give advice on upgrading. I remember getting an email from v0.18 to v0.19. But couldn't find a page that might suggest some thing like.


TASK [specs : Redirect to https] ****************************************************************************************************************
skipping: [192.241.137.219] => changed=false 
  skip_reason: Conditional result was False

TASK [specs : uri] ******************************************************************************************************************************
fatal: [192.241.137.219]: FAILED! => changed=false 
  connection: close
  content: |-
    <html>
    <head><title>502 Bad Gateway</title></head>
    <body bgcolor="white">
    <center><h1>502 Bad Gateway</h1></center>
    <hr><center>nginx/1.10.3 (Ubuntu)</center>
    </body>
    </html>
  content_length: '182'
  content_type: text/html
  date: Fri, 19 Jul 2019 13:27:32 GMT
  elapsed: 0
  msg: 'Status code was 502 and not [200]: HTTP Error 502: Bad Gateway'
  redirected: false
  server: nginx/1.10.3 (Ubuntu)
  status: 502
  url: http://127.0.0.1

PLAY RECAP **************************************************************************************************************************************
192.241.137.219            : ok=77   changed=54   unreachable=0    failed=1    skipped=29   rescued=0    ignored=0 

Capistrano: "database.yml does not exist"

Trying out the new capistrano installer. Again, thank you for continuing to improve this awesome installer.

I'm seeing this:

00:02 deploy:check:linked_files
      ERROR linked file /var/www/consul/shared/config/database.yml does not exist

I don't know enough about the use of the shared folder. Did I miss a step? Thanks!

Capistrano RVM task looks in root

Hello, not sure if I've configured the installer incorrectly, but Capistrano seems to be checking the /root/ directory for rvm:

cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as [email protected]: bundle exit status: 102
bundle stdout: Can not find rvm in '/root/.rvm'.
bundle stderr: Nothing written

I think this is in the deploy user's:

/home/deploy/.rvm

Any ideas?

Make the playbook inmutable

What

If the script is only run once and everything goes smoothly there is no problem...
However if something goes wrong, some roles are not prepared to be run a second time

We should make the playbook inmutable, thus being able to run the playbook multiple times without seeing errors due t changes in the state of the server

Checkout:

500 | Internal server error on /valuation page

After deployment when I try tot access the valuation page in the admin panel I get a 500 internal server error.

Edit: This only happens when there are no participatory budget present.

Use secrets.yml to store SMTP credentials

What

  • Update CONSUL's secrets.yml.example to include SMTP credentials
  • Update installer correspondingly to set this variables in the production server's secrets.yml

Why

  • Right now the installer is hardcoding these values in the production.rb file, but with a new deploy these values will be overwritten by the default production.rb file

Run in production environment by default

What

Right now the configuration is setup to run in Rails's development environment.
Instead, run all necessary commands and setups in the production environment by default

Dockerized deployment

Hi all,
Thanks for the good work.
I'm used to deploying docker images in production (AWS or CleverCloud) and to have a docker-compose file similar to my production env (1 postgresql and at least 2 app instances).
I'm a bit lost in this setup.

Do you know if someone is working on a production docker image of consul ?
I guess, we would need a way to enable cron and emails sender on only one instance.
To create and migrate automatically the database schema/data.

Regards,
Thomas.

Cant rerun playbook

So regarding issue #57 I got stuck when running the playbook. I run the playbook with the method mentiond in issue #14. Now however I have another issue which keeps me from rerunning the playbook. On a fresh installed server I can run the playbook fine. But when I try to re run the playbook I get the following error:

TASK [unicorn : Kill running process] *****************************************************************************************************************************************************
changed: [127.0.0.1] => (item=22359)

TASK [unicorn : Start Unicorn] ************************************************************************************************************************************************************
fatal: [127.0.0.1]: FAILED! => {"changed": true, "cmd": "/home/deploy/.rvm/gems/ruby-2.3.2/wrappers/unicorn -c config/unicorn.rb -E production -D", "delta": "0:00:01.548114", "end": "2018-11-21 15:08:24.682874", "msg": "non-zero return code", "rc": 1, "start": "2018-11-21 15:08:23.134760", "stderr": "master failed to start, check stderr log for details", "stderr_lines": ["master failed to start, check stderr log for details"], "stdout": "", "stdout_lines": []}
	to retry, use: --limit @/root/installerold/consul.retry

However when I try to run the playbook for a third time it will work again. so it does for the fifth, seventh etc. But for the fourth and sixth time it gives me the error again. And so it goes on. All the even number of tries will fail. The only file I have changed is the group_vars/all.

Deploy production and preproduction in the same server

Hi Consul commnunity!

I want to display production and preproduction on the same server. I can install the production environment on the server, but when I try to do the same with the preproduction environment I get an error.

error-consul-preproduction

Capistrano's task is aborted, and he does not continue to deploy.

deploy-secrets.yml

preproduction:
  deploy_to: "/home/deploy/preproduction-consul"
  ssh_port: "22"
  server1: "my-server-ip"
  db_server: "localhost"
  user: "deploy"
  server_name: "my-web1.com"
  full_app_name: "preproduction-consul"

production:
  deploy_to: "/home/deploy/consul"
  ssh_port: "22"
  server1: "my-server-ip"
  db_server: "localhost"
  user: "deploy"
  server_name: "my-web.com"
  full_app_name: "consul"

Production and preproduction have to be on the same server. But the domains of the application are different.

If you need anything else, let me know.

Thanks!

Update README to run playbook in verbose mode

What

When running the playbook locally in virtual box, the verbose option is active
However when running in a remote server, we need to add the -v option when running the playbook

Update documentation to run the playbook for remote users in verbose mode by default

Establish Log files permissions

When trying to run bin/rails c development an error message appeared (I lost it, my bad) that was saying something like "no permissions to write log files". So I did sudo chmod +664 *.log at /log folder.

This should be done by the installer script

ERROR 500 | Creating new budget investment proposal

Hey!,

Regarding issue #70 I have SSL working now but when creating a new proposal for a budget investment I receive a 500 error in browser. The new proposal is created anyway.

This is what I found in the log file:

W, [2019-01-17T09:42:28.265116 #23197]  WARN -- : [ActiveJob] [Rollbar] Got unexpected status code from Rollbar api: 400
W, [2019-01-17T09:42:28.484569 #23197]  WARN -- : [Rollbar] Got unexpected status code from Rollbar api: 400
F, [2019-01-17T09:42:28.488860 #23197] FATAL -- :
SocketError (getaddrinfo: Name or service not known):
  app/controllers/budgets/investments_controller.rb:62:in `create'

Playbook is stuck

I used the method in issue #14 to install consul on my Ubuntu server. The first time i could run the playbook without any problem. Only when I try to run the playbook again it gets stuck on a certain task.
It gets stuck on:

TASK [rails : Load configuration seeds] **************************************************************************
task path: /root/installer/roles/rails/tasks/main.yml:65
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: root
<127.0.0.1> EXEC /bin/sh -c 'echo ~root && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1542197596.54-5055084817189 `" && echo ansible-tmp-1542197596.54-5055084817189="` echo /root/.ansible/tmp/ansible-tmp-1542197596.54-5055084817189 `" ) && sleep 0'
Using module file /usr/lib/python2.7/dist-packages/ansible/modules/commands/command.py
<127.0.0.1> PUT /root/.ansible/tmp/ansible-local-32731ATOFOF/tmpLysohr TO /root/.ansible/tmp/ansible-tmp-1542197596.54-5055084817189/AnsiballZ_command.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1542197596.54-5055084817189/ /root/.ansible/tmp/ansible-tmp-1542197596.54-5055084817189/AnsiballZ_command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1542197596.54-5055084817189/AnsiballZ_command.py && sleep 0'

Rerun the playbook after activating the Capistrano role

Refereces

Comment: #57 (comment)

Context

The installer runs fine multiple times with the default setup, but if the capistrano role is activated, the second time the installer is run it raises an error. This is one the error messages that have been reported:

TASK [rails : Deploy user permissions] *****************************************
fatal: [178.128.162.212]: FAILED! => {"changed": false, "gid": 0, "group": "root", "mode": "0644", "msg": "chown failed: [Errno 1] Operation not permitted: '/home/deploy/consul/app/views/pages/.conditions.html.erb.swp'", "owner": "root", "path": "/home/deploy/consul/app/views/pages/.conditions.html.erb.swp", "size": 1024, "state": "file", "uid": 0}

Objectives

Allow the installer to run multiple times with the Capistrano role activated.

Restarting the Server leaves the application dead

We followed your install steps and magically the app runned. We restarted the server and the app was not running. This steps is supposed to be for production environments. Restarting the server shouldn't leave the application killed. Is there a way to keep a service running the app alive?

The install scripts does so many hidden jobs, the person that deploys the app is hard to understand. It might be easy for ruby people to understand it, but for me that i am a 10 years none ruby developer seems complicated. It used to be more clear what you should do to install for example a wordpress site. I don't know if ruby makes the things more complicate, but i would expect to grab some files, configure the db, configure ngnix/ruby and Boom the website appears.

We didn't also find any mention in the documentations for the update procedure. Does it exist?

thank you in advance for clarifying our questions.

HOWTO: Successful consul ansible installation in Ubuntu 16.04 locally

Hi,

In order to install consul with this ansible script locally in Ubuntu 16.04, one needs to do the following step-by-step:

  1. Login as root user (sudoers did not work)

  2. Create a rsa key
    # ssh-keygen -t rsa

  3. Install necessary dependency for 4 below:
    # apt-get install software-properties-common -y

  4. Install latest ansible and dependencies from upstream (requires ansible 2.4+ which is not readily available in ubuntu repository)
    # add-apt-repository ppa:ansible/ansible
    # apt-get update
    # apt-get -y install ansible curl python-simplejson git

  5. Pull this repo (as root itself) and change to the installer directory:
    # git clone https://github.com/consul/installer
    # cd installer

  6. Disable remote_user as it is a local ansible instance:
    # sed -i -e 's/remote_user: vagrant/#remote_user: vagrant/g' consul.yml

  7. Make hosts to reflect localhost (127.0.0.1)
    # echo '[servers]' > hosts
    # echo '127.0.0.1' >> hosts

  8. Make changes to /root/installer/group_vars/all to reflect your own needs.

  9. Run ansible-playbook with -c local flag to run locally.
    # ansible-playbook consul.yml -i hosts -c local --extra-vars "target=servers"

  10. Once finished, browse to the server_ip_address in a browser and login with default credentials:

username: [email protected]
password: 12345678

Once logged in, change admin password. That's all!

Hope this helps for anyone who shall be in my shoes.

Kudos goes to the entire consul team for developing such a thought-provoking software and a comprehensive installer.

Cheers,
/z

Task Install gems Fails

Scenario, libs and versions :

Local Machine:
Ubuntu 16.04.4 LTS
ruby 2.3
usuario local: user
simplejson 3.8.1
ansible 2.5.4

Remote server:
Ubuntu 16.04.4 LTS
ruby 2.3
user: deploy (created by ansible )
python-simplejson 3.8.1
Ubuntu 16.04 x64

TASK [capistrano : Install gems (this may take a few minutes)] *********************************************************************************************************************************************
fatal: [10.99.52.80]: FAILED! => {"changed": true, "cmd": "gem install bundler && bundle install", "delta": "0:00:17.052788", "end": "2018-06-27 12:06:38.257560", "msg": "non-zero return code", "rc": 5, "start": "2018-06-27 12:06:21.204772", "stderr": "", "stderr_lines": [], "stdout": "Successfully installed bundler-1.16.2\nParsing documentation for bundler-1.16.2\nDone installing documentation for bundler after 3 seconds\n1 gem installed\nDon't run Bundler as root. Bundler can ask for sudo if it is needed, and\ninstalling your bundle as root will break this application for all non-root\nusers on this machine.\nFetching gem metadata from https://rails-assets.org/..\nFetching gem metadata from https://rubygems.org/...........\nFetching gem metadata from https://rails-assets.org/..\nUsing rake 12.3.1\nUsing concurrent-ruby 1.0.5\nUsing i18n 0.9.1\nUsing minitest 5.11.1\nUsing thread_safe 0.3.6\nUsing tzinfo 1.2.4\nUsing activesupport 4.2.10\nUsing builder 3.2.3\nUsing erubis 2.7.0\nUsing mini_portile2 2.3.0\nFetching nokogiri 1.8.2\nInstalling nokogiri 1.8.2 with native extensions\nGem::Ext::BuildError: ERROR: Failed to build gem native extension.\n\n current directory: /var/lib/gems/2.3.0/gems/nokogiri-1.8.2/ext/nokogiri\n/usr/bin/ruby2.3 -r ./siteconf20180627-6654-k2t6v1.rb extconf.rb\nmkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h\n\nextconf failed, exit code 1\n\nGem files will remain installed in /var/lib/gems/2.3.0/gems/nokogiri-1.8.2 for\ninspection.\nResults logged to\n/var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/nokogiri-1.8.2/gem_make.out\n\nAn error occurred while installing nokogiri (1.8.2), and Bundler cannot\ncontinue.\nMake sure that gem install nokogiri -v '1.8.2' succeeds before bundling.\n\nIn Gemfile:\n graphiql-rails was resolved to 1.4.8, which depends on\n rails was resolved to 4.2.10, which depends on\n actionmailer was resolved to 4.2.10, which depends on\n actionpack was resolved to 4.2.10, which depends on\n actionview was resolved to 4.2.10, which depends on\n rails-dom-testing was resolved to 1.0.9, which depends on\n nokogiri", "stdout_lines": ["Successfully installed bundler-1.16.2", "Parsing documentation for bundler-1.16.2", "Done installing documentation for bundler after 3 seconds", "1 gem installed", "Don't run Bundler as root. Bundler can ask for sudo if it is needed, and", "installing your bundle as root will break this application for all non-root", "users on this machine.", "Fetching gem metadata from https://rails-assets.org/..", "Fetching gem metadata from https://rubygems.org/...........", "Fetching gem metadata from https://rails-assets.org/..", "Using rake 12.3.1", "Using concurrent-ruby 1.0.5", "Using i18n 0.9.1", "Using minitest 5.11.1", "Using thread_safe 0.3.6", "Using tzinfo 1.2.4", "Using activesupport 4.2.10", "Using builder 3.2.3", "Using erubis 2.7.0", "Using mini_portile2 2.3.0", "Fetching nokogiri 1.8.2", "Installing nokogiri 1.8.2 with native extensions", "Gem::Ext::BuildError: ERROR: Failed to build gem native extension.", "", " current directory: /var/lib/gems/2.3.0/gems/nokogiri-1.8.2/ext/nokogiri", "/usr/bin/ruby2.3 -r ./siteconf20180627-6654-k2t6v1.rb extconf.rb", "mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h", "", "extconf failed, exit code 1", "", "Gem files will remain installed in /var/lib/gems/2.3.0/gems/nokogiri-1.8.2 for", "inspection.", "Results logged to", "/var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/nokogiri-1.8.2/gem_make.out", "", "An error occurred while installing nokogiri (1.8.2), and Bundler cannot", "continue.", "Make sure that gem install nokogiri -v '1.8.2' succeeds before bundling.", "", "In Gemfile:", " graphiql-rails was resolved to 1.4.8, which depends on", " rails was resolved to 4.2.10, which depends on", " actionmailer was resolved to 4.2.10, which depends on", " actionpack was resolved to 4.2.10, which depends on", " actionview was resolved to 4.2.10, which depends on", " rails-dom-testing was resolved to 1.0.9, which depends on", " nokogiri"]}
to retry, use: --limit @/home/jesux/projects/installer/consul.retry

Configure system email service

What

Configure the installer to use sendmail instead of SMTP by default or configure postfix so the server can send emails automatically.

Alternatively, we could configure postfix on the server during the installation process.

Why

Until we configure delayed job properly, we're getting a nasty 500 error when trying to send emails using the installer's default SMTP settings.

Clarify workflow and differences between the Installer and Capistrano

Context

Some users are executing the Installer again to upload new versions of CONSUL to the production servers.

Objectives

Clarify in the documentation that the installer is only meant to me run once.
Future changes to the code in the production server should be done using Capistrano

Notes

CONSUL updates for new Releases should also be done using Capistrano.

However, we still have to decide how to deal with system updates. For the moment, system administrators are expected to do this job, but maybe in the future the Installer can handle upgrading system libraries.

Capistrano Taks Fails

Scenario, libs and versions :

Local Machine:
Ubuntu 16.04.4 LTS
ruby 2.3
usuario local: user
simplejson 3.8.1
ansible 2.5.4

Remote server:
Ubuntu 16.04.4 LTS
ruby 2.3
user: deploy (created by ansible )
python-simplejson 3.8.1
Ubuntu 16.04 x64

TASK [capistrano : Copy deploy.rb (locally)] ***************************************************************************************************************************************************************
fatal: [10.99.52.80]: FAILED! => {"changed": false, "checksum": "df14b6d8a11dd21c701813c32e1da2b6548c5e69", "dest": "/home/user/projects/installer/consul/config/deploy.rb", "gid": 1000, "group": "user", "mode": "0644", "msg": "chgrp failed: failed to look up group wheel", "owner": "deploy", "path": "/home/user/projects/installer/consul/config/deploy.rb", "size": 1922, "state": "file", "uid": 1001}
to retry, use: --limit @/home/user/projects/installer/consul.retry

PLAY RECAP *************************************************************************************************************************************************************************************************
10.99.52.80 : ok=47 changed=3 unreachable=0 failed=1

Staging environment

What

Add an option to run the playbook for a staging environment

Why

To be able to deploy changes to a staging environment using cap staging/preproduction deploy before deploying the changes to a production server

How

Add a variable environment and substitute the hardcoded production words and files in the installer

Notes

The current CONSUL codebase uses preproduction instead of staging environment
It's probably simpler to go ahead and use preproduction for now πŸ‘Œ

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.