intermine / docker-intermine-gradle Goto Github PK
View Code? Open in Web Editor NEWDocker for InterMine
License: GNU Lesser General Public License v2.1
Docker for InterMine
License: GNU Lesser General Public License v2.1
It always returns unhealthy.
After installing tomcat we need to configure
JAVA_OPTS="$JAVA_OPTS -Dorg.apache.el.parser.SKIP_IDENTIFIER_CHECK=true"
to allow the template page working properly.
The configuration here seems correct and
docker exec intermine_tomcat echo "$JAVA_OPTS"
return
-Dorg.apache.el.parser.SKIP_IDENTIFIER_CHECK=true
which is correct BUT the template page (and the query builder constraint popup) do not work (empty page) and in the log we have the javax.el.ELException.
We have in the instructions to not include the trailing slash in their data directory parameter. People are going to forget or not read an include the trailing slash.
Can we instead just ignore the trailing slash?
We talked about this already a bit.
Example:
tigerMine
.~/git/tigermine
<source name="go" type="go">
<property name="src.data.file" location="/data/go-annotation/go-basic.obo"/>
</source>
/data/go-annotation/
How can I then use the docker files to build my tigerMine?
Sometimes when intermine_builder runs build.sh, an unstable connection can cause it to fail.
We should try and be more lenient to these failures, and automatically retry a few times.
Can we expose the messages written to the console during the build?
for biotestmine it's fine but if we wanted to use this on a real intermine build that is much longer, it would be good to know what's going on! :)
(lower priority)
Is this:
Doing the same thing as this:
https://github.com/intermine/docker-intermine-gradle/blob/master/README.md#set-environment-variables
Maybe we could delete that last section? Or are they setting the ENV variables locally and for docker so we need both?
I tried to follow the Quickstart in the README and I'm unable to start a biotestmine instance when running docker-compose -f local.docker-compose.yml up --build --force-recreate
or docker-compose -f dockerhub.docker-compose.yml up
. Here's the error I get when running the latter command.
intermine_builder | FAILURE: Build failed with an exception.
intermine_builder |
intermine_builder | * What went wrong:
intermine_builder | Execution failed for task ':dbmodel:retrieveSingleSource'.
intermine_builder | > dataDir is not a directory:uniprot
intermine_builder |
intermine_builder | * Try:
intermine_builder | Run with --info or --debug option to get more log output. Run with --scan to get full insights.
intermine_builder |
intermine_builder | * Exception is:
intermine_builder | org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':dbmodel:retrieveSingleSource'.
intermine_builder | at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:110)
intermine_builder | at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:77)
intermine_builder | at org.gradle.api.internal.tasks.execution.OutputDirectoryCreatingTaskExecuter.execute(OutputDirectoryCreatingTaskExecuter.java:51)
intermine_builder | at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:59)
intermine_builder | at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:54)
intermine_builder | at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:59)
intermine_builder | at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:101)
intermine_builder | at org.gradle.api.internal.tasks.execution.FinalizeInputFilePropertiesTaskExecuter.execute(FinalizeInputFilePropertiesTaskExecuter.java:44)
intermine_builder | at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:91)
intermine_builder | at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:62)
intermine_builder | at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:59)
intermine_builder | at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54)
intermine_builder | at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
intermine_builder | at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34)
intermine_builder | at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.run(EventFiringTaskExecuter.java:51)
intermine_builder | at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:300)
intermine_builder | at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:292)
intermine_builder | at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:174)
intermine_builder | at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
intermine_builder | at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
intermine_builder | at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:46)
intermine_builder | at org.gradle.execution.taskgraph.LocalTaskInfoExecutor.execute(LocalTaskInfoExecutor.java:42)
intermine_builder | at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareWorkItemExecutor.execute(DefaultTaskExecutionGraph.java:273)
intermine_builder | at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareWorkItemExecutor.execute(DefaultTaskExecutionGraph.java:258)
intermine_builder | at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:135)
intermine_builder | at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:130)
intermine_builder | at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.execute(DefaultTaskPlanExecutor.java:200)
intermine_builder | at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.executeWithWork(DefaultTaskPlanExecutor.java:191)
intermine_builder | at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.run(DefaultTaskPlanExecutor.java:130)
intermine_builder | at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
intermine_builder | at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
intermine_builder | at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
intermine_builder | Caused by: : dataDir is not a directory:uniprot
intermine_builder | at org.intermine.task.DirectoryConverterTask.execute(DirectoryConverterTask.java:63)
intermine_builder | at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293)
intermine_builder | at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
intermine_builder | at org.intermine.plugin.integrate.IntegrateUtils$_closure5.doCall(IntegrateUtils.groovy:128)
intermine_builder | at org.intermine.plugin.integrate.IntegrateUtils.invokeMethod(IntegrateUtils.groovy)
intermine_builder | at org.intermine.plugin.integrate.IntegrateUtils$_closure2.doCall(IntegrateUtils.groovy:51)
intermine_builder | at org.intermine.plugin.integrate.IntegratePlugin$_apply_closure7$_closure20.doCall(IntegratePlugin.groovy:179)
intermine_builder | at org.gradle.api.internal.AbstractTask$ClosureTaskAction.execute(AbstractTask.java:724)
intermine_builder | at org.gradle.api.internal.AbstractTask$ClosureTaskAction.execute(AbstractTask.java:697)
intermine_builder | at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.run(ExecuteActionsTaskExecuter.java:131)
intermine_builder | at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:300)
intermine_builder | at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:292)
intermine_builder | at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:174)
intermine_builder | at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
intermine_builder | at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
intermine_builder | at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:120)
intermine_builder | at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:99)
intermine_builder | ... 31 more
intermine_builder |
intermine_builder |
intermine_builder | * Get more help at https://help.gradle.org
intermine_builder |
intermine_builder | BUILD FAILED in 7s
intermine_builder | Fri Dec 6 10:28:53 UTC 2019
intermine_builder |
intermine_builder | finished
intermine_builder |
intermine_builder |
intermine_builder | ERROR: 256
intermine_builder | failed with exit code 0: /home/intermine/intermine/biotestmine/gradlew --stacktrace --no-daemon integrate -Psource=uniprot-malaria
intermine_builder |
intermine_builder | check log: pbuild.log
intermine_builder exited with code 1
@leoank Do you have a chance to look at this?
Note: I haven't done any setup other than installing docker and docker-compose and starting the docker daemon, so let me know if I've missed anything.
intermine_postgres | LOG: checkpoints are occurring too frequently (4 seconds apart)
intermine_postgres | HINT: Consider increasing the configuration parameter "checkpoint_segments".
We recommend this setting:
checkpoint_segments | 128
Is there an easy way to update the config of the postgres in the docker container?
so in my humanmine.properties for example, I have this:
db.common-tgt-items.datasource.serverName=localhost
db.common-tgt-items.datasource.databaseName=items-humanmine
db.common-tgt-items.datasource.user=ank
db.common-tgt-items.datasource.password=ank
I have to update the username and password to be postgres
Instead we could have the script delete any lines that start with:
# delete these lines from humanmine.properties
db.common-tgt-items.datasource.user
db.common-tgt-items.datasource.password
And replace with these lines
# add these lines to humanmine.properties
db.common-tgt-items.datasource.user=postgres
db.common-tgt-items.datasource.password=postgres
Same for tomcat.
Just less steps for the user, and less bugs!
It would be useful to have a Github action which when a PR is merged, or change committed to master branch, builds the docker containers and if there are no errors, pushes the new images to https://hub.docker.com/u/intermine/.
The docker commands for doing this once auth secrets have been added: (filling in imagename
and commithash
)
docker build -t imagename path/to/Dockerfile
docker tag imagename intermine/imagename:latest
docker tag imagename intermine/imagename:commithash
docker push intermine/imagename
For the sake of automation, I think using the commit hash instead of a semantic version is okay.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.