How to add additional php modules? about service-cli HOT 11 CLOSED

@JensvdHeydt Seems like a thread I should comment on! I wondered myself what the best way to extend Docksal's stock service images, and the most non-intrusive/maintainable method I could think of was:

• Create your own Docker image project.
• Use whatever version of Docksal's service you want to use in the Dockerfile.
• Make your additions/subtractions (see the Dockerfile reference).
• Build the image and make sure it's add to Docksal's machine.
• To test and verify that your built image works with Docksal, see Docksal's documentation on extending docksal.yml and update the service's image value with whatever the name of your service is.
• After you're satisfied, push up your image project to Github or Bitbucket, make it public.
• On Docker Hub, create an Automated Build project. Link your Github/Bitbucket account and point it to your image repository.
• Under Build Settings, specify the branch you want to create a build off of and what the image tag should be called.
• In the Linked Repositories section, specify docksal/<service>. This way, every time Docksal updates that service image on Docker Hub, your image will automatically be rebuilt as well based on their latest changes.
• If you think your changes should be part of their stock image, fork their project and make your changes and submit a PR as I did in #8.

Using this method you can extend Docksal's service containers, let them do their own thing, while simultaneously doing your own thing using their own thing, and staying up to date with their own thing without any additional effort from you doing your own thing.

Use case

I needed to support Behat in the CLI container service. I know Docksal provides their own Behat container, but it expects a certain structure, and I have a BLT project which uses a close but slightly different file structure. So I opted to just run behat via the CLI service. I had to add the lsot and default-jre packages via apt-get to get it working. I did the following:

• Created my own image project, tested and verified it worked as I expected https://bitbucket.org/lpeabody/blt-cli/commits/95b1e4521c2d855bbe27e73ce49e020f4d61dc4e.
• Created an Automated Build on Docker Hub and linked to my repo, and setup the build to trigger whenever docksal/cli is updated. https://hub.docker.com/r/lpeabody/blt-cli/builds/ NOTE I have not triggered a single build, it was done automatically because docksal/cli was updated.
• I'm using the Acquia stack that Docksal provides, so my docksal.env looks like:

DOCKSAL_STACK="acquia"

• And I updated my docksal.yml to pull my lpeabody/blt-cli image as opposed to docksal/cli:

version: "2.1"

services:
  cli:
    image: lpeabody/blt-cli

• Run fin up. I'm done. My project is now pulling my image directly from Docker Hub, and it's automatically kept up to date the with the latest changes introduced by docksal/cli.

EDIT: There is an interim step you need to run to pull the latest changes from Docker Hub, fin up won't do it automatically (unless the image hasn't already been pulled). If the image is already present in Docksal's machine, and you want to pull the latest changes to it from Docker Hub, you have to run fin docker pull <image name>:<tag>! It's after you've pulled the latest from Docker Hub that you want to run fin up, as was indicated in the last step above.

That's a lot

It is but it isn't. It's actually a very simple process and fairly painless. The hardest part is building the Dockerfile to adequately extend which ever of Docksal's services. Hope this helps.

from service-cli.

For those who are wondering about @lmakarov's third-to-last paragraph re using the "build" directive so you don't have to fork and publish an alternative image. Here's an example of using it to install and enable additional apache modules (mod_proxy).

1. Edit .docksal/docksal.yml. Under "web" replace "image: docksal/web:1.0-apache2.2" with "build: ${PROJECT_ROOT}/.docksal/web"
2. Create ".docksal/web/Dockerfile" with something like this:

FROM docksal/web:1.0-apache2.2

# Install packages
RUN DEBIAN_FRONTEND=noninteractive apt-get update && \
    DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes install \
    libapache2-mod-proxy-html \
    libxml2-dev && \
# Enable Apache modules.
    a2enmod proxy_html proxy_http proxy_balancer

# Overwrite default virtual hosts
COPY drupal.conf /etc/apache2/sites-available/drupal.conf
COPY drupal-ssl.conf /etc/apache2/sites-available/drupal-ssl.conf

# Launching apache2 directly (instead of apachectl) so it gets PID 1
CMD ["apache2", "-DFOREGROUND"]

3. For custom VirtualHost config, copy drupal.conf and drupal-ssl.conf from https://github.com/docksal/service-web/tree/master/apache2.2/config/apache2/sites-available to your .docksal/web and customize. If you don't need custom vhost config you can omit this step and remove the COPY commands from the .docksal/web/Dockerfile.

4. Run fin docker-compose build web and fin up (you need to do this whenever Dockerfile changes).

from service-cli.

@lpeabody That's was a very detailed and clear writeup! I'm thinking to copy and paste it into our docs or maybe a guest blog post (planning on launching a Docksal blog soon) ;)

from service-cli.

@krlucas come back home please.

On a more serious note, this is awesome, thanks for the tip. Definitely useful on a per-project basis, though if you find yourself using it consistently on more than one project, probably best to setup a build project on Docker Hub.

from service-cli.

The docksal/cli image does not currently support this via some sort of built-in configuration options, which would be applied at boot. This is totally possible, just has not been a priority.

If an extension or a feature is something more or less mainstream, then we'll be happy to include it in the stock image. Here's an example of a PR, that has been just merged yesterday to include the php7-bz2 extension - #8

I believe php7.0-redis is a pretty common extension, that we can add.

Another option is to extend Docksal images via a custom per-service Dockerfile, which would be stored within every project. There's already a feature request for that here: docksal/docksal#171

What you can do today is manually installing anything you want (including arbitrary php extensions) via a project's init script. So you can add there something like this:

sudo apt-get update
sudo apt-get install php7.0-redis

from service-cli.

Small modification for clarity. Leonid meant

fin exec sudo apt-get update
fin exec sudo apt-get install php7.0-redis

from service-cli.

@lmakarov, thanks for the quick and very helpful answer. I ended up adding a single line to my init - command as you proposed. Works good enough for me. :-)

fin exec 'sudo apt-get update && sudo apt-get -y --force-yes --no-install-recommends install php7.0-redis'

from service-cli.

FYI I went ahead and include the redis php extension in both php5 and php7 image versions.
The new edge images (docksal/cli:edge and docksal/cli:edge-php7) will be available on Docker Hub within 1h.

You can switch your stack to use those images and run fin update --project-images to update the stack. Or, wait until we get this out in a stable release within the next few days.

from service-cli.

@lmakarov Thanks! It could probably use some editing, but I am game for a guest blog post collaboration. I love Docksal so I'm happy contributing to it in any way possible.

from service-cli.

@lpeabody Thats a great writeup. Thank you very much!

I do like your solution and as I'm still learning about docker and docker-compose I came to realize that many docker images support environment variables to configure them. Wouldn't it be practical to introduce such a variable to list additional php-modules or other apt-packes that one wants to be installed? Just asking to clarify.

from service-cli.

@JensvdHeydt

Wouldn't it be practical to introduce such a variable to list additional php-modules or other apt-packes that one wants to be installed?

While technically possible, this would introduce a provisioning phase at boot time for an image. That goes agains what Docker images are all about - consistency. If you downloaded an image you have everything it needs pre-packaged and no more stuff to download or install. This can also increase the cold boot time for an image substantially.

For certain things it does make sense to configured them at boot via environment variables. docksal/cli currently supports the following:

• XDEBUG_ENABLED - enable/disable xdebug
• HOST_UID and HOST_GID - to match the docker user uid:gid in the container to the one of the host's user (this resolved permission issues on files touched/created within the container).

We could pre-package more modules in the stock image and just disable them, then have ENV variables to enabled them as necessary. That's an option, however... This bloats the image with non-mainstream modules.

In addition to the two approaches already covered here (init script and a forked image), there is a 3rd one.

Docker Compose supports the build directive, where you can specify a custom Dockerfile stored within the project's code base. Compose will build a local version of the image when you start the project the first time, then reuse that image again and again, until you update the Dockerfile and ask docker-compose to rebuild.

This approach can obviously be less consistent than the pre-build forked image approach @lpeabody described earlier, however it is also less complex to setup and more reliable and straightforward compared to using init scripts for extra provisioning.

I think this 3rd option would be a good middle group in terms of complexity vs consistency.

from service-cli.