islandora-devops / isle-dc Goto Github PK
View Code? Open in Web Editor NEWISLE 8 - Dockerized Islandora 8 Deployment orchestrated with docker-compose
License: MIT License
ISLE 8 - Dockerized Islandora 8 Deployment orchestrated with docker-compose
License: MIT License
MVP3 - Accept latest alpaca PR islandora-deprecated/isle-alpaca#3
Currently the official Tomcat base image doesn't come with a ROOT.war or tomcat-admin panel. Is this wanted or needed?
Hi,
The company I work for would like to start working with ISLE 8 ASAP. Is there some way we can help speed up the development? Is there a backlog of work? Can we do a small conference call to talk things through?
Ik would like to hear from you!
Thanks!
Running make
fails with error:
ERROR: for blazegraph Cannot start service blazegraph: error while mounting volume '/var/lib/docker/volumes/islandora_blazegraph-data/_data': failed to mount local volume: mount /c/Users/Dev/projects/isle-dc/data/blazegraph/data:/var/lib/docker/volumes/islandora_blazegraph-data/_data, flags: 0x1000: no such file or directory
Encountered errors while bringing up the project.
make: *** [Makefile:21: up] Error 1
changing the volume type to default (rather than local
) fixes the issue. As no other volumes have type local
, it probably makes sense to bring the blazegraph mount in line line with the others.
After ingesting a large tiff open seadragon shows a 500 error.
Unable to open [object Object]: HTTP o attempting to load TileSource
First time visiting a site that implements an insecure https request will cause the media to load within Open Seadragon.
Quick solution: is to go directly to https://islandora-isle-dc.12-34-56-78.traefik.me/cantaloupe/iiif/2 and accept the warning. This should probably be addressed, there is no obvious/simple indicator to the developer/user as to why the failure is happening.
From #21 (comment)
Re-testing the current setup has revealed now that the current wodby image and the Drupal build are having the following issues.
Drupal Report Status now indicating You are using an incompatible Solr schema
Solr path needs to be /
not /solr
.
This feels like something changed in the Drupal settings / code level. Putting back on the TO DO list for us to test / fix.
If you checkout an existing Drupal site into codebase
and do make
, the build will fail with
Composer could not find a composer.json file in /app
To initialize a project, please create a composer.json file as described in the https://getcomposer.org/ "Getting Started" section
make: *** [Makefile:15: drupal_init] Error 1
When I run make
in the development branch, make runs and generates the expected files. When I run it in the master branch (commit e48f215), this happens:
mark@mark-ThinkPad-X1-Carbon-6th:~/Documents/hacking/isle-dc$ make
make: *** No targets specified and no makefile found. Stop.
Any suggestions?
This environment is intended for local development. Users will create a Drupal site in the folder codebase, which gets bind mounted into the drupal service container. Allowing developers to use composer / drush locally to work out of the codebase folder.
CODEBASE links to https://github.com/Islandora-Devops/isle-dc/blob/development/codebase
[STUB] Please build this out for steps for user testing
Please note Cary Gordon is also interested in testing this and should be included.
Drush 9.0+ is in the "Requirements" list. This is on the host where you're running docker-compose
?
[STUB] Please build this out for steps for user testing
Docker-compose is ignoring values set during the makemake/docker-compose up process. Not sure any other values are being ignored.
docker-compose.env.yml & docker-compose.yml have these values prior and after the make/docker-compose up process.
DRUPAL_DEFAULT_ACCOUNT_EMAIL: [email protected]
DRUPAL_DEFAULT_ACCOUNT_PASSWORD: abc1234567890
Observed the admin user has "[email protected]" with "password" as its password.
Once working remove wodby tags etc from docker-compose and envs.
Per @nikathone's comment here #6
"The drupal.Dockerfile still using registry.gitlab.com/nikathone/drupal-docker-good-defaults/php-nginx:latest for php and nginx base image but this will change once we have a CI to build images/nginx-php/Dockerfile."
@nikathone Can we make an official Islandora version of this post-merge? We now have control over the Islandora Dockerhub account to be able to push images. We can also make a new repo accordingly if need be too. Your thoughts?
[STUB] Please build this out for steps for user testing
[STUB] Please build this out for steps for user testing
cp sample.env to .env
./scripts/drupal/init.sh --codebase islandora
docker-compose pull
docker-compose up -d
docker-compose exec drupal drush uli (once everything is done)
Cleanup and remove old readmes accordingly
For the future once the Make file is back in play update readmes accordingly
BONUS: Update with all involved repos especially the https://github.com/Born-Digital-US/drupal-project
Review Makefile again once we get build working with MVP3 and 2 merged.
Ensure that value of docker_compose_project ?=
can be changed from hard-coded islandora to match .env value instead otherwise it appears Docker volumes and container names don't match and cause issues
Review @nigelgbanks 's suggestion here #6 (comment) re Minor non-issue. Should be added to .PHONY targets.
Update README or documentation with syntax in Make file, benefits of using and scripts / actions it assumes.
Define SSL usage process for LE, commercial and self-signed certs in non-local environments
Move fcrepo and fcrepo related microservices to separate docker-compose.yml includes
This allows people to not have to use it when setting up for the first time. Please note the fcrepo & gemini dbs will still be created if only using Drupal.
[STUB] Please build this out for steps for user testing
If you don't know to change docker-compose.drupal.mariadb.yml this will bite you. There is no documentation pointing out this file needs to be change but is needed if assigning an IP or domain.
mariadb-${COMPOSE_PROJECT_NAME-isle-dc}.${DRUPAL_SITE_HOST-traefik.me}
mariadb-${COMPOSE_PROJECT_NAME-isle-dc}-${DRUPAL_SITE_HOST-traefik.me}
---------------------------------------^
Toronto is still coming up as default timezone not UTC for the newly created Drupal site when resetting admin password.
@birkland https://github.com/Islandora-Devops/isle-dc/blob/development/docker-compose.yml#L243 appears we are still using a private image for the alpaca connector?
Should we be building from https://github.com/Islandora-Devops/isle-alpaca/blob/dev/Dockerfile and pushing to the official Islandora Dockerhub now? I can then update the docker-compose.yml
accordingly.
(would love to also assign you here but still can't seem to, are you a member yet? I can send an invite again)
I can do this but just want to be sure we're on the right track.
Additionally @nikathone has some good thoughts and ideas about work using the drupal image entry-point.sh which he can either open up a new ticket for or add here for further discussion as needed.
When running:
docker-compose -p islandora down \
--rmi local \
--volumes \
--remove-orphans
or
docker-compose -p islandora up --remove-orphans --detach
Is there any issue to removed unneeded settings from the main docker-compose.yml
?
From #6 (review)
One minor fix for the reversion/
"@nikathone This looks good except on line 41 of the docker-compose.yml, should this section of the drupal service that is currently:
depends_on:
drupal_db:
condition: service_healthy
actually be changed to:
depends_on:
mariadb:
condition: service_healthy"
From @marksandford on Slack:
"I've gotten the test ingest files ready to go and zipped up. Instructions are in the Readme, but it's basically installed as a module and a series of drush commands are run.
One caveat is that I consistently get SQLSTATE[40001]: Serialization failure
when running the media migration. Running it again with --update seems to fix it (as described in Readme). Setting mysql to read-committed avoids the problem entirely, but I have no idea what the implications are for that in production, nor do I know if the ISLE config will differ in ways that will avoid the error"
@dannylamb Have you run into this issue yet?
I've been running a local setup on my Mac OSX (Catalina) and running into extremely long page load times. The Manage->Extend page takes just shy of 30 seconds to load. Many pages take at least 4 seconds to load. It's a pretty awful developer experience.
@birkland researched adding the delegated
flag to the volume mount for the website codebase. It seems promising and I'll submit a PR for that in a few minutes for other folks to try out.
Atm we have all the site urls follow this pattern SERVICE_NAME.COMPOSER_PROJECT_NAME.localhost
however this causes issues when the site is hosted on a machine other than localhost
for Chrome, as Chrome does a bit of magic for localhost urls that the other browsers don't do. Likely the only solution here is to drop .localhost
from our urls and just have SERVICE_NAME.COMPOSER_PROJECT_NAME
as the urls by default.
@dannylamb Should we rebuild and clear this prior to merge work?
Advise on how to incorporate elements / services/ config changes from recent Islandora upgrade from 1.0.0 to 1.1.0 https://gist.github.com/dannylamb/fc910f4d1730f21191f384d533502e24
This should wait until we have the full stack working again. (4.30.2020) per Weekly meeting standup.
From PR #16
Add this to the README.md
Depends on https://github.com/Islandora-Devops/isle-alpaca/pull/3
Comment in the build sections for and point to ^^ once it's merged (or pull it in and point to a local copy on disk).
## How to Test
From your isle-dc folder:
- `docker-compose down -v`
- `rm -rf codebase && mkdir codebase` to clean the codebase
- If you haven't already done so, `(cd scripts; ./generate_keys.sh)` to generate keys
- `docker-compose up --build -d`
- `docker exec -it isle_dc_proto_solr bash -c "solr create_core -c islandora"`
- `docker exec -it isle_dc_proto_php bash -c "sh /scripts/islandora/install-islandora.sh"`
You can test by ingesting a repository item, giving it a media and tagging that media as an original file. This should trigger derivatives.
Admittedly, there's still something super wonky with karaf/camel uploading derivatives, and sometimes you get uploaded files of 0. I'm still working through that, but you can at least see that the whole system is functioning, albeit a bit impaired.
MVP3 Blazegraph needs an install script to be added to the soon to be merged in Makefile for this service to start up.
Create gemini, milliner and recast PRs - Danny (for Monday if possible)
Update the https://github.com/Islandora-Devops/isle-dc/ with the appropriate services as needed
Review and update code from Gavin on the following new fcrepo related microservices, issue PRs as needed.
After importing an object via migration_islandora_csv I attempted to edit an object. After doing so I get an empty manifest and a 500 error on the object's page.
Manifest
{
"@type": "sc:Manifest",
"@id": "http://islandora-isle-dc-12-34-56-78.traefik.me/node/839/manifest",
"label": "IIIF Manifest",
"@context": "http://iiif.io/api/presentation/2/context.json",
"sequences": [{
"@context": "http://iiif.io/api/presentation/2/context.json",
"@id": "http://islandora-isle-dc-12-34-56-78.traefik.me/node/839/sequence/normal",
"@type": "sc:Sequence"
}]
}
Something interesting I saw in watchdog. 2 calls but one called the traefik.me URL and the other localhost.
openseadragon Request for IIIF manifest at https://islandora-isle-dc-12-34-56-78.traefik.me/node/840/manifest returned 0
openseadragon Request for IIIF manifest at http://localhost:8000/node/840/manifest returned 0
Merge MVP2a, 2b & 3 #6
Gavin to resolve merge conflicts in PR
I think there is remaining work to add SYN and get it working in the isle-fcrepo image?
@dannylamb Apologies if you've already done this but I don't see any commits per se?
Compared to an install via islandora-playbook, the composer.json file from
Born-Digital-US/drupal-project is missing a couple dependencies:
"solarium/solarium": "^5.1",
"symfony/event-dispatcher": "4.3.4 as 3.4.99",
For now, it seems the best solution is to update the Born-Digital-US/drupal-project.
Suggestion: docker-compose.override.yml to set passwords and other options. Have MAKE file generate it to give a simplified file to use.
https://devilbox.readthedocs.io/en/latest/configuration-files/docker-compose-override-yml.html
Please feel free to close if this isn't a good idea.
From the April 23 weekly call, @nigelgbanks raised the idea of discussing potentially using s6-overlay for
The task should be to identify which images and services could benefit from utilizing s6-overlay to help with process management especially child processes handling as well as piping logs to stdout (as needed)
Also reference this issue: Logging to stderr / stdout for all images / containers / services #45
Users need to manually add the domain to the hosts file as before but only for windows users.
Update activemq with configuration fix for login
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.