Git Product home page Git Product logo

docker-intermine-gradle's People

Contributors

22poojagaur avatar heralden avatar leoank avatar

Stargazers

avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

interminebot heralden rplanel niveditarufus 22poojagaur sabreen-parveen edgegreen vitaliyzasada anvgithub tetiana19 anqluck bioinformapping kaacyberpro kakituapp wormbase panjunwei manabuishii

docker-intermine-gradle's Issues

easier to build - set the username and pw in the build

so in my humanmine.properties for example, I have this:

db.common-tgt-items.datasource.serverName=localhost
db.common-tgt-items.datasource.databaseName=items-humanmine
db.common-tgt-items.datasource.user=ank
db.common-tgt-items.datasource.password=ank

I have to update the username and password to be postgres

Instead we could have the script delete any lines that start with:

# delete these lines from humanmine.properties
db.common-tgt-items.datasource.user
db.common-tgt-items.datasource.password

And replace with these lines

# add these lines to humanmine.properties
db.common-tgt-items.datasource.user=postgres
db.common-tgt-items.datasource.password=postgres

Same for tomcat.

Just less steps for the user, and less bugs!

write to console during build?

Can we expose the messages written to the console during the build?

for biotestmine it's fine but if we wanted to use this on a real intermine build that is much longer, it would be good to know what's going on! :)

(lower priority)

Continuous deployment to dockerhub

It would be useful to have a Github action which when a PR is merged, or change committed to master branch, builds the docker containers and if there are no errors, pushes the new images to https://hub.docker.com/u/intermine/.

The docker commands for doing this once auth secrets have been added: (filling in imagename and commithash)

docker build -t imagename path/to/Dockerfile
docker tag imagename intermine/imagename:latest
docker tag imagename intermine/imagename:commithash
docker push intermine/imagename

For the sake of automation, I think using the commit hash instead of a semantic version is okay.

Property -Dorg.apache.el.parser.SKIP_IDENTIFIER_CHECK=true seems not working

After installing tomcat we need to configure
JAVA_OPTS="$JAVA_OPTS -Dorg.apache.el.parser.SKIP_IDENTIFIER_CHECK=true"
to allow the template page working properly.
The configuration here seems correct and
docker exec intermine_tomcat echo "$JAVA_OPTS"
return
-Dorg.apache.el.parser.SKIP_IDENTIFIER_CHECK=true
which is correct BUT the template page (and the query builder constraint popup) do not work (empty page) and in the log we have the javax.el.ELException.

how to build a mine that's not biotestmine?

We talked about this already a bit.

Example:

  1. I want to make tigerMine.
  2. I run make_mine to generate a new InterMine. This script creates ~/git/tigermine
  3. I update project XML to have a single source:
    <source name="go" type="go">
      <property name="src.data.file" location="/data/go-annotation/go-basic.obo"/>
    </source>
  1. I download the file go-basic.obo and place it in /data/go-annotation/

How can I then use the docker files to build my tigerMine?

Failure when following quickstart

I tried to follow the Quickstart in the README and I'm unable to start a biotestmine instance when running docker-compose -f local.docker-compose.yml up --build --force-recreate or docker-compose -f dockerhub.docker-compose.yml up. Here's the error I get when running the latter command.

intermine_builder    | FAILURE: Build failed with an exception.
intermine_builder    | 
intermine_builder    | * What went wrong:
intermine_builder    | Execution failed for task ':dbmodel:retrieveSingleSource'.
intermine_builder    | > dataDir is not a directory:uniprot
intermine_builder    | 
intermine_builder    | * Try:
intermine_builder    | Run with --info or --debug option to get more log output. Run with --scan to get full insights.
intermine_builder    | 
intermine_builder    | * Exception is:
intermine_builder    | org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':dbmodel:retrieveSingleSource'.
intermine_builder    | 	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:110)
intermine_builder    | 	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:77)
intermine_builder    | 	at org.gradle.api.internal.tasks.execution.OutputDirectoryCreatingTaskExecuter.execute(OutputDirectoryCreatingTaskExecuter.java:51)
intermine_builder    | 	at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:59)
intermine_builder    | 	at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:54)
intermine_builder    | 	at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:59)
intermine_builder    | 	at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:101)
intermine_builder    | 	at org.gradle.api.internal.tasks.execution.FinalizeInputFilePropertiesTaskExecuter.execute(FinalizeInputFilePropertiesTaskExecuter.java:44)
intermine_builder    | 	at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:91)
intermine_builder    | 	at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:62)
intermine_builder    | 	at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:59)
intermine_builder    | 	at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54)
intermine_builder    | 	at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
intermine_builder    | 	at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34)
intermine_builder    | 	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.run(EventFiringTaskExecuter.java:51)
intermine_builder    | 	at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:300)
intermine_builder    | 	at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:292)
intermine_builder    | 	at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:174)
intermine_builder    | 	at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
intermine_builder    | 	at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
intermine_builder    | 	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:46)
intermine_builder    | 	at org.gradle.execution.taskgraph.LocalTaskInfoExecutor.execute(LocalTaskInfoExecutor.java:42)
intermine_builder    | 	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareWorkItemExecutor.execute(DefaultTaskExecutionGraph.java:273)
intermine_builder    | 	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareWorkItemExecutor.execute(DefaultTaskExecutionGraph.java:258)
intermine_builder    | 	at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:135)
intermine_builder    | 	at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:130)
intermine_builder    | 	at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.execute(DefaultTaskPlanExecutor.java:200)
intermine_builder    | 	at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.executeWithWork(DefaultTaskPlanExecutor.java:191)
intermine_builder    | 	at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.run(DefaultTaskPlanExecutor.java:130)
intermine_builder    | 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
intermine_builder    | 	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
intermine_builder    | 	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
intermine_builder    | Caused by: : dataDir is not a directory:uniprot
intermine_builder    | 	at org.intermine.task.DirectoryConverterTask.execute(DirectoryConverterTask.java:63)
intermine_builder    | 	at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293)
intermine_builder    | 	at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
intermine_builder    | 	at org.intermine.plugin.integrate.IntegrateUtils$_closure5.doCall(IntegrateUtils.groovy:128)
intermine_builder    | 	at org.intermine.plugin.integrate.IntegrateUtils.invokeMethod(IntegrateUtils.groovy)
intermine_builder    | 	at org.intermine.plugin.integrate.IntegrateUtils$_closure2.doCall(IntegrateUtils.groovy:51)
intermine_builder    | 	at org.intermine.plugin.integrate.IntegratePlugin$_apply_closure7$_closure20.doCall(IntegratePlugin.groovy:179)
intermine_builder    | 	at org.gradle.api.internal.AbstractTask$ClosureTaskAction.execute(AbstractTask.java:724)
intermine_builder    | 	at org.gradle.api.internal.AbstractTask$ClosureTaskAction.execute(AbstractTask.java:697)
intermine_builder    | 	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.run(ExecuteActionsTaskExecuter.java:131)
intermine_builder    | 	at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:300)
intermine_builder    | 	at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:292)
intermine_builder    | 	at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:174)
intermine_builder    | 	at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
intermine_builder    | 	at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
intermine_builder    | 	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:120)
intermine_builder    | 	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:99)
intermine_builder    | 	... 31 more
intermine_builder    | 
intermine_builder    | 
intermine_builder    | * Get more help at https://help.gradle.org
intermine_builder    | 
intermine_builder    | BUILD FAILED in 7s
intermine_builder    | Fri Dec  6 10:28:53 UTC 2019
intermine_builder    | 
intermine_builder    | finished
intermine_builder    | 
intermine_builder    | 
intermine_builder    | ERROR: 256
intermine_builder    | failed with exit code 0: /home/intermine/intermine/biotestmine/gradlew --stacktrace --no-daemon integrate -Psource=uniprot-malaria
intermine_builder    | 
intermine_builder    | check log: pbuild.log
intermine_builder exited with code 1

@leoank Do you have a chance to look at this?

Note: I haven't done any setup other than installing docker and docker-compose and starting the docker daemon, so let me know if I've missed anything.

Handle connection timeouts by retrying

Sometimes when intermine_builder runs build.sh, an unstable connection can cause it to fail.

We should try and be more lenient to these failures, and automatically retry a few times.

disregard the trailing slash somehow

We have in the instructions to not include the trailing slash in their data directory parameter. People are going to forget or not read an include the trailing slash.

Can we instead just ignore the trailing slash?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.