Hello,

I'm trying to know where to add in my app.yml 2 lines of config for MariaDB conf (basically put in my.cnf)

[mysqld]
collation-server = utf8mb4_unicode_ci 
character-set-server = utf8mb4     

Thank you for your help :-)

FOO = bar -> FOO=bar

We took time to cleanup newest config templates(by removing already defaulted values), now it's time to take care of oldest templates such as 5, 5.6 and 7.0

Starting from ansible 2.4, we can replace manual systemctl daemon-reload call by:

- name: just force systemd to reread configs (2.4 and above)
  systemd: daemon_reload=yes

See: http://docs.ansible.com/ansible/latest/systemd_module.html

Should at least be applied to manala.systemd/handlers/main.yml

Add always_run = yes (deprecated) or check_mode = no to clean task.

Config

manala_postgresql_version: 9.4
manala_postgresql_config_template: config/default.dev.j2
manala_postgresql_config_hba_template: config/hba/default.dev.j2

manala_postgresql_roles:
  - role: app
    password:   ~
    attributes: ['SUPERUSER']

Result

TASK [manala.postgresql : requirements > Check if version is defined.] *********
skipping: [localhost]

TASK [manala.postgresql : install > Packages] **********************************
changed: [localhost] => (item=[u'postgresql-9.4', u'postgresql-contrib-9.4'])

TASK [manala.postgresql : config > PostgreSQL configuration] *******************
changed: [localhost]

TASK [manala.postgresql : config > PostgreSQL host-based authentication (hba) configuration] ***
changed: [localhost]

TASK [manala.postgresql : services > Ensure postgresql is running] *************
ok: [localhost] => (item=postgresql)

TASK [manala.postgresql : roles > Requirements] ********************************
ok: [localhost]

TASK [manala.postgresql : roles > Handle] **************************************
failed: [localhost] (item={u'attributes': [u'SUPERUSER'], u'password': None, u'role': u'app'}) =>
{
  "failed":true,
  "item":{
    "attributes":[
      "SUPERUSER"
    ],
    "password":null,
    "role":"app"
  },
  "msg":"unable to connect to database: fe_sendauth: no password supplied\n"
}

In https://github.com/manala/ansible-roles/blob/master/manala.oauth2-proxy/tasks/install.yml the oauth2-proxy role attempts to install a debian package using apt. The oauth2-proxy binary isn't available as a debian package. The project's homepage says that there are two ways to install the package, either by downloading a pre-built binary or by downloading the source and compiling locally, https://github.com/bitly/oauth2_proxy#installation

Right here: https://github.com/manala/ansible-roles/blob/master/manala.environment/lookup_plugins/manala_environment_variables.py#L9

Hello,

After passing the right variable:

app_local_patterns:
  apt_repositories:
    - blackfire

  php_blackfire: false

  php_blackfire_agent:
    - server-id:  xxxx
    - server-token: xxxx
  php_blackfire_client:
    - client-id:  xxxx
    - client-token: xxxx

The blackfire agent is installed as reported when running make provision BUT it fails when it tries to start the service (or restart).

Solution was to connect to VM, use blackfire-agent register to manually insert the token and server id.

And after doing so, re-run the make provision

See: #133

[WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ item.when }}

Should be easy to fix :)

File : ansible-roles/manala.influxdb/templates/config/base.conf.j2

Issue : Configuration template seems to be wrong for :

• line 393 : [[collectd]]
• line 428 : [[opentsdb]]

It seems that we have to remove the double [[ ]] to keep simple []

Hey there,

Is there a built-in way to uninstall old PHP in case of an upgrade ?

manala.php doesn't seem to support both PHP version

Or maybe it's not documented, but what about having php5-fpm && php7-fpm ? Is this supported by manala.php ?

Thanks.

Since you merged #24 4hours ago, the repository https://github.com/manala/ansible-role-php is not yet updated.

• manala.alternatives

TASK [manala.alternatives : selections > Alternatives]
 [WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ manala_alternatives_selections|length }}

• manala.apt

TASK [manala.apt : update > Upgrade dist]
 [WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ manala_apt['update']|default(false) }}

TASK [manala.apt : repositories > Exclusive - remove files]
 [WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ manala_apt_repositories_exclusive and ( item not in __manala_apt_repositories_results.results |map(attribute='item') |map(attribute='source')
|map('regex_replace', '^deb (\[.+\] )?https?:\/\/([^ ]+)[ ].*$', '\2') |map('replace', '.', ' ') |map('replace', '/', ' ') |map('trim') |map('replace', ' ', '_') |map('replace', '-', '_') |map('regex_replace', '(.*)', '\1.list') |list ) }}

TASK [manala.apt : udate > Backup package marks]
 [WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ manala_apt['update']|default(false) }}

TASK [manala.apt : udate > Hold packages]
 [WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ manala_apt['update']|default(false) }}

TASK [manala.apt : udate > Restore package marks]
 [WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ manala_apt['update']|default(false) }}

• manala.deploy

TASK [manala.deploy : include]
 [WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ item.when }}

TASK [manala.deploy : tasks/command > "touch /tmp/manala_deploy_tasks.txt"]
 [WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ item.when }}

• manala.docker

TASK [manala.docker : update > Pull]
 [WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ manala_docker['update']|default(false) }}

• manala.grafana

TASK [manala.grafana : dashboards > List dashboards]
[DEPRECATION WARNING]: Supplying headers via HEADER_* is deprecated and will be removed in a future version. Please use `headers` to supply headers for the request.
This feature will be removed in a future release. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.

• manala.npm

TASK [manala.npm : update > Packages]
 [WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ manala_npm['update']|default(false) }}

• manala.php

TASK [manala.php : extensions > Enabled]
 [WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ lookup( 'manala_php_extensions', manala_php_extensions, manala_php_versions[manala_php_version|string], wantenabled=true, wantmap=true, wantlist=true )|length
}}

TASK [manala.php : extensions > Disabled]
 [WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ lookup( 'manala_php_extensions', manala_php_extensions, manala_php_versions[manala_php_version|string], wantenabled=false, wantmap=true, wantlist=true )|length
}}

Late hello,

FAILED! => {"changed": false, "failed": true, "msg": "Your \"manala_php_version\" is not compatible with your release"}

Trying to install php 7.1 on Debian 9 (stretch).

Thanks :-)

Hello,

MySQL 8.0 has been released on April 2018. It would be awesome of it would be possible to install it with manala.

Hi.
Got an error during provisioning new EC2:

TASK [manala.logrotate : configs > Templates] ************************************************************************************************************************************************************************************************
failed: [dev.monitor.domain.tld] (item={'file': 'nginx', 'config': [{'/var/log/nginx/.log': [{'size': '100M'}, 'missingok', {'rotate': 5}, 'compress', {'delaycompress': None}, 'notifempty', {'create': '0640 www-data adm'}, 'sharedscripts', 'daily', 'postrotate systemctl reload nginx.service', 'endscript']}]}) => {"changed": false, "item": {"config": [{"/var/log/nginx/.log": [{"size": "100M"}, "missingok", {"rotate": 5}, "compress", {"delaycompress": null}, "notifempty", {"create": "0640 www-data adm"}, "sharedscripts", "daily", "postrotate systemctl reload nginx.service", "endscript"]}], "file": "nginx"}, "msg": "AnsibleUndefinedVariable: 'dict object' has no attribute 'iteritems'"}

Weird a bit, as have your role used for another environment on the previous week and all was good...

Playbook content:

    - role: manala.logrotate
      manala_logrotate_configs:
        - file: nginx
          config:
            - /var/log/nginx/*.log:
              - size: 100M
              - missingok
              - rotate: 5
              - compress
              - delaycompress:
              - notifempty
              - create: 0640 www-data adm
              - sharedscripts
              - daily
              - postrotate
                  systemctl reload nginx.service
              - endscript

Python on the host: 2.7.13
Ansible version: ansible 2.5.4

provide state: latest in manala_apt_packages

Build status banners provided by TravisCI takes a lots of times to be loaded. The repo homepage displaying the README.md file takes around 10s to be fully loaded because of that.

I suggest to move them into a dedicated file or to find a better way to display them.

Non exhaustive and non checked list:

• phpmyadmin
• oauth2-proxy
• httpie
• ...

Currently the manala role allows only to go to version 1.7 of elasticsearch, but the version 5 is out and the 6 is on its way

https://www.elastic.co/fr/downloads/elasticsearch
https://www.elastic.co/guide/en/elasticsearch/reference/6.0/setup-upgrade.html

As spotted by @chalasr, to be applied on all related roles:

raise AnsibleError('Expect a dict')-> raise AnsibleError('Expected a dict')
raise AnsibleError('Expect "code" key')-> raise AnsibleError('Missing "code" key')

IPV6 adoption is a global effort and handling shorewall6 could help

When changing shorewall.conf using for example:

manala_shorewall_config:
   - BLACKLIST: "NEW,INVALID,UNTRACKED,ESTABLISHED"
   - DISABLE_IPV6: "Yes"

shorewall is not restarted and thus changes are not applied.

• manala.deploy - 0600_writable.yml

fatal: [debian]: FAILED! => {"changed": true, "cmd": ["goss", "--gossfile", "0600_writable.goss.yml", "validate"], "delta": "0:00:00.072518", "end": "2017-10-06 10:46:23.951592", "failed": true, "rc": 1, "start": "2017-10-06 10:46:23.879074", "stderr": "", "stderr_lines": [], "stdout": ".F.\n\nFailures/Skipped:\n\nFile: /tmp/app/current/logs: mode:\nExpected\n    <string>: 0755\nto equal\n    <string>: 0775\n\nTotal Duration: 0.002s\nCount: 3, Failed: 1, Skipped: 0", "stdout_lines": [".F.", "", "Failures/Skipped:", "", "File: /tmp/app/current/logs: mode:", "Expected", "    <string>: 0755", "to equal", "    <string>: 0775", "", "Total Duration: 0.002s", "Count: 3, Failed: 1, Skipped: 0"]}

-> Fixed by #107

• manala.docker - 0300_applications.yml

fatal: [debian]: FAILED! => {"changed": true, "cmd": ["goss", "--gossfile", "0300_applications.goss.yml", "validate"], "delta": "0:00:10.933164", "end": "2017-10-06 11:01:35.869854", "failed": true, "rc": 1, "start": "2017-10-06 11:01:24.936690", "stderr": "", "stderr_lines": [], "stdout": "FFF\n\nFailures/Skipped:\n\nCommand: hello-world: exit-status:\nExpected\n    <int>: 125\nto equal\n    <int>: 0\nCommand: hello-world: stdout: patterns not found: [Hello from Docker!]\n\nCommand: npm --version: exit-status:\nExpected\n    <int>: 125\nto equal\n    <int>: 0\n\nTotal Duration: 10.821s\nCount: 3, Failed: 3, Skipped: 0", "stdout_lines": ["FFF", "", "Failures/Skipped:", "", "Command: hello-world: exit-status:", "Expected", "    <int>: 125", "to equal", "    <int>: 0", "Command: hello-world: stdout: patterns not found: [Hello from Docker!]", "", "Command: npm --version: exit-status:", "Expected", "    <int>: 125", "to equal", "    <int>: 0", "", "Total Duration: 10.821s", "Count: 3, Failed: 3, Skipped: 0"]}

-> Fixed by #108

See: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=865762

Tests have been adapted according to this bug. Different options:

1. Waiting for an hypothetical backport in stable branch
2. Force default to /bin/sh in role
3. Do nothing...

In all cases, tests MUST be adapted in consequence.

Pygmalion is old, unmaintained, and buggy (see: ohmyzsh/ohmyzsh#2317)

We need to choose another one.

https://github.com/robbyrussell/oh-my-zsh/wiki/themes

J'ai trouvé ca:

• https://zshthem.es/browse-zsh-themes/
• https://www.slant.co/topics/7553/~theme-for-oh-my-zsh
• https://hashnode.com/post/what-is-your-favorite-oh-my-zsh-theme-cirwop8fb0h0gdu53bc8rmeiv

• Remove disable_systemd tests pre_tasks
• Mongodb on jessie don't support sysv scripts anymore
• Mysql-wsrep-server-5.7 on jessie|stretch don't support sysv scripts anymore
• Sensu services tests on jessie/stretch
• Thumbor services tests
• Docker on stretch don't support sysv scripts anymore (a sysv script is manually installed during stretch tests)
• Varnish services tests
• Systemd service itself
• MariaDB 10.4 https://github.com/manala/ansible-roles/pull/411/files
• Php service & blackfire service

It has been done for symfony/symfony in symfony/symfony#22461 and I find it nice. Output: https://travis-ci.org/symfony/symfony/jobs/223214166

Given the output for a role is quite verbose, could we fold all green roles for cross-roles PRs? See e.g. https://travis-ci.org/manala/ansible-roles/jobs/231465081
I can pick it if you like it

We are just waiting for upstream support right here: http://rep.logentries.com/dists/index.html

• Update security repository url from http://security.debian.org/ to http://security.debian.org
• Httpredir is now deprecated, we must use deb.debian.org

See:

• manala/docker-images#14
• https://github.com/manala/packer-template-skeleton/issues/3

See: https://the.exa.website/

In update task, instead of calling dpkg --[g|s]et-selections manually

Switch:

• Wheezy -> Dotdeb
• Jessie -> Sury

See somewhere around https://github.com/manala/ansible-role-skeleton/blob/master/vars/main.yml#L69

Should we copy from "/usr/share/zoneinfo//" or create a symbolic link ???

Grafana provides pre-packaged deb file, can we add a variable check (like when manala_grafana_deb_url is defined) to install from external deb url? (Becuase the upstream version in ubuntu is pretty old, for example: 2.6 in 16.04)

Hello & thanks, phpmyadmin version is 4.6.6deb4 and there are some bugs because of new features with php 7.2. Is it possible to upgrade phpmyadmin to > 4.7.4 (latest 4.7.7) ?

phpmyadmin 4.7.4 is supposed to have "Fixed several compatibility issues with PHP 7.2" https://www.phpmyadmin.net/news/2017/8/24/phpmyadmin-474-released/

For example:

  <Directory /path/to/uploads>
    <Limit ALL>
      DenyAll
    </Limit>

    <Limit CDUP CWD PWD XCWD XCUP>
      AllowAll
    </Limit>

    <Limit STOR STOU>
      AllowAll
    </Limit>
  </Directory>

• import_tasks/include_tasks

[DEPRECATION WARNING]: The use of 'include' for tasks has been deprecated. Use 'import_tasks' for static inclusions or 'include_tasks' for dynamic inclusions. This feature will be removed in a future release. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
[DEPRECATION WARNING]: include is kept for backwards compatibility but usage is discouraged. The module documentation details page may explain more about this rationale.. This feature will be removed in a future release. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.

• meta/min_ansible_version

Discussion sur le deploy d'un build local

Ceci est le compte-rendu du mes essaies (fructueux) de deploy d'un build local dans le cadre d'un projet dont l'hébergement n'est pas maitrisé et ne disposant pas des outils de build nécessaires (node, yarn/npm, composer, ...)

Créer le build

Pour cela j'ai repris ce qui avait été fait sur Grip, à savoir simplement utiliser manala.deploy pour deploy en local.

Cela ce traduit par ajouter un host local ainsi qu'un groupe qu'on nommera release dans le fichier de hosts :

localhost_demo ansible_connection=local
localhost_prod ansible_connection=local

[release_demo]
localhost_demo

[release_prod]
localhost_prod

[release:children]
release_demo
release_prod

On ajoute donc deux cible à notre Makefile :

## Release application (demo)
release@demo:
	ansible-playbook ansible/release.yml --inventory-file=ansible/hosts --limit=release_demo

## Release application (prod)
release@prod:
	ansible-playbook ansible/release.yml --inventory-file=ansible/hosts --limit=release_prod

ainsi que le fichier de configuration group_vars/release.yml et ces deux déclinaisons group_vars/release_prod.yml et group_vars/release_demo.yml.

Dans ses fichiers pas, pas de surprise, on reprend la configuration d'un deploy classique, comme sur un serveur distant. On fera seulement attention au répertoire où est déployé notre build. Pour ma part j'ai mis ça dans /tmp :

manala_deploy_dir: "/tmp/build"

Notre deploy local sera joué par un playbook release.yml qui se appellera le rôle manala.deploy suivit d'un task créant l'archive au forma souhaité (un tar.gz ici) :

---
- hosts: release
  any_errors_fatal: true
  roles:
    - manala.deploy
  tasks:
    - name: Release > Remove previous archive
      file:
        path: /tmp/build/current.tar.gz
        state: absent
    - name: Release > Create archive
      shell: "tar zcvf /tmp/build/current.tar.gz ."
      args:
        chdir: /tmp/build/current
        creates: current.tar.gz

⚠️ Attention à bien créer l'archive depuis la racine de build afin de ne pas embarquer d'arborescence (cf chdir avant le tar).

ℹ️ Vous aurez remarqué je j'utilise une command shell pour créer mon archive est pas un module ansible. C'est tout simplement parce que le module archive n'est disponible que depuis ansible 2.3 qui vient tous juste de sortir et que pour l'instant nous sommes bloqué en 2.2 (paquet debian oblige).

Et voilà, vous avez maintenant une belle archive qui contient le build de votre projet.

Deploy de l'archive

Le déploiement de l'archive pourrait se faire juste en poussant l'archive et en l'extrayant. Mais je voulais garder ce système de releases/shared/current dans une soucis de simplicité et d'habitude.

Le deploy se passe donc comme un deploy classique, mais avec une nouvelle stratégie (manala/ansible-role-deploy#22) remplaçant le classique git clone. Cette stratégie utilise le module unarchive de Ansible qui permet de pousser une archive local sur le serveur puis de l'extraire à l'endroit souhaité. L'avantage est que le module ansible gère tout seul l'upload, le fichier temporaire de l'archive, son extraction et sa destruction.

Seul modif à faire dans votre config de deploy :

manala_deploy_strategy: unarchive
manala_deploy_strategy_unarchive_src: "/tmp/build/current.tar.gz"

Alternative rsync

Au début je souhaitait me passer de l'archive et déployer en rsync grâce à la stratégie déjà existante synchronize. Seulement cela nécessite que rsync soit installé sur le serveur de prod, ce qui n'était pas le cas sur mon projet test. Mais c'est faisable. Un simple scp n'est malheureusement pas possible (du moins via le module copy de Ansible qui poserais de gros soucis de performance lors d'un nombre de fichier important).

De plus rsync n'a que peux d'avantage ici puisque l'on conserve la notion de release sur le serveur. On repart donc de zero à chaque releases, pas de diff.

Conclusion

Je pense qu'il y a encore un peu de travail pour rentre le truc un peu plus automatique. Peut être en créer une tache gérant la création facultative d'une archive en fin de deploy (local).

A voir également si on peut pousser le truc en faisant un meta playbook et une meta config automatisant ce système de création de build puis déploiement de celui ci sans avoir à créer deux playbook (l'un pour le deploy, l'autre pour le build).

Mais dans un premier temps, je ne trouve pas ça soit problèmatique que les deux taches soit séparer. Peut être juste mettre la tache release en dépendence de la tache deploy dans le make file suffirait.

Check elasticsearch is running in tests using https://www.elastic.co/guide/en/elasticsearch/guide/current/running-elasticsearch.html#CO1-2