bio4j / angulillos Goto Github PK

Java 8 library for strongly typed graph data

License: GNU Affero General Public License v3.0

Scala 0.97% Java 99.03%

bio4j traversal java schema tinkerpop graph-schema graph graphs graph-database data-model

angulillos's Introduction

Bio4j bioinformatics graph data platform

Bio4j is a bioinformatics graph data platform, integrating most data available in Uniprot KB (SwissProt + Trembl), Gene Ontology (GO), UniRef (50,90,100), NCBI Taxonomy, and Expasy Enzyme DB.

Bio4j provides a completely new and powerful framework for protein related information querying and management. The use of a graph-based data model makes possible to store and query data in a way that semantically represents its own structure. On the contrary, traditional relational models and databases must flatten the data they represent into tables, creating artificial ids in order to connect the different tuples; which can in some cases eventually lead to domain models that have almost nothing to do with the actual structure of data.

Project structure and overview

Bio4j can look a bit intimidating at first, with all those repositories with kind of similar names; here you have a guided tour around:

bio4j/bio4j

In this repository bio4j/bio4j you will find the generic Bio4j model and API. Entities, relationships and their properties are modeled using a typed property graph model. For example, there are vertex types for Protein or GoTerm, and a GoAnnotation edge type going from Protein to GoTerm. This graph schema is separated into different graphs, corresponding to the different data sources (UniProt, Go, UniRef, ...) and connections between them (UniProtGo, UniProtUniRef, ...).

The API, based on bio4j/angulillos, lets you write generic typed traversals over this graph schema:

protein.uniref50Member_outV()
  .map(
    UniRef50Cluster::uniRef50Member_inV
  )
  .map(
    prts -> prts.map(
      Protein::goAnnotation_outV
    )
  );

which can later be executed on a particular backend. Generic data import code is also here, which can be used to load the data using any implementation of angulillos.

bio4j/angulillos

You can think of bio4j/angulillos as a strongly typed version of the property graph model. You can describe graph schemas and write generic traversals over them which are guranteed to be well-typed in that for example

you cannot retrieve the outgoing edges of and edge
and you can get the tweets that a user tweeted, but not the users that a tweet follows!

bio4j/bio4j-titan

In bio4j/bio4j-titan you will find a Titan-based Bio4j distribution. This is the the default standard distribution, and we also provide through AWS S3 the database binaries with all data already loaded. Go there if you want to stop reading and use Bio4j now!

bio4j/angulillos-titan

bio4j/angulillos-titan is an implementation of the angulillos API using Titan.

Documentation

General docs: docs/
Code docs: docs/src/
API Docs: v0.12.0

Community and contact

Gitter chat The easiest and fastest way to contact developers and ask for help
bio4j-user Google group / mailing list
@bio4j Twitter
Bio4j LinkedIn

Licensing

Bio4j is an open source platform released under the AGPLv3 license.

angulillos's People

Contributors

Stargazers

Watchers

Forkers

digideskio laughedelic

angulillos's Issues

experiment with typed Tinkerpop3 traversals

examples - several graphs with deps

Better names for arities

Most of this is being discussed in #40

how to specify graphs with dependencies

In the spirit of what is done in Bio4j

Restructure methods for adjacent edges/vertices

I have rough an idea on how to improve all these inOneE, inOptionalE, inManyE, etc. For that we'll need to make arities useful:

split arity on two: in-arity and out-arity (there already are separate traits for this)
add representation type to arity (E, Optional<E>, Stream<E>)
put those arity specific in/out methods to the arity traits
call them from the typed graph/vertex interfaces

Something like

    sealed trait Arity { 

      type C[E]
      def narrow(edges: Stream[E]): C[E]
    }

    trait ExactlyOne extends Arity { type C[E] = E; ... }
    trait AtMostOne  extends Arity { type C[E] = Option[E]; ... }
    trait ManyOrNone extends Arity { type C[E] = Stream[E]; ... }


    trait InArity extends Arity
    trait OutArity extends Arity

    trait FromExactlyOne extends InArity with ExactlyOne
    trait FromAtMostOne  extends InArity with AtMostOne
    trait FromManyOrNone extends InArity with ManyOrNone

    trait ToExactlyOne extends OutArity with ExactlyOne
    trait ToAtMostOne  extends OutArity with AtMostOne
    trait ToManyOrNone extends OutArity with ManyOrNone

then in the TypedVertex:

  default <
    // src
    S extends TypedVertex<S,ST,SG,I,RV,RE>,
    ST extends TypedVertex.Type<S,ST,SG,I,RV,RE>,
    SG extends TypedGraph<SG,I,RV,RE>,
    // rel
    R extends TypedEdge<S,ST,SG, R,RT,RTA, RG,I,RV,RE, N,NT,G>,
    RT extends TypedEdge.Type<S,ST,SG, R,RT,RTA, RG,I,RV,RE, N,NT,G>,
    RTA extends TypedEdge.Type.Arity<...>,       // some magic here, please???
    RG extends TypedGraph<RG,I,RV,RE>
  >
  RTA<E> inE(RT relType) { return relType.arity.narrow(graph().inE(relType, self()); }

@eparejatobes WDYT?
Sorry for using scala here, but I think you got the idea. Do you think it's doable in Java?

Split interfaces on read/write-only

I would split typed and untyped graph interfaces on parts with

writing methods (addVertex/addEdge, setProperty, commit)
reading methods (the rest)
and probably indexes-related stuff

In the writing part we can add general methods for schema initialization. Then it's used for importing data.

The reading part is used for working with an instance of already imported data.

@eparejatobes WDYT?

Problem when using last version published

Now the following piece of code:

TitanNode.Type

gives the following error:

Error:(55, 34) java: com.ohnosequences.typedGraphs.titan.TitanNode.Type is not public in com.ohnosequences.typedGraphs.titan.TitanNode; cannot be accessed from outside package

Is that normal?

are access modifiers for raw too wide?

Bio4j induced requirements

@pablopareja I did some general clean up and implemented some extra stuff for Titan; all this is in 0.2.0-SNAPSHOT. Please use this issue for feature requests that you surely will be identifying as the work in bio4j/bio4j-titan#14 progresses. The idea is that you shouldn't need to use anything from the raw Titan API.

add full support for Titan indexes

See

Advanced schema graph

Not only creating types/labels in the DB, but also creating a real graph, i.e. connecting DB-specific edge-type presentation with its source/target types with an edge, etc. So that you can query the DB for the schema structure.

In TitanDB this is straightforward, because EdgeLabel implements TitanVertex. This needs some investigation for other possible backends.

test with sample data for Titan

What I started for GO could work

Problem with Index stuff in TypedGraphs

I just ran into this:

SEVERE: Exception retrieving protein P0DM41
Sep 04, 2014 10:38:32 AM com.bio4j.model.uniprot.programs.ImportUniprot importUniprot
SEVERE: null
Sep 04, 2014 10:38:32 AM com.bio4j.model.uniprot.programs.ImportUniprot importUniprot
SEVERE: com.ohnosequences.typedGraphs.titan.TitanTypedVertexIndex$Default.query(TitanTypedVertexIndex.java:67)
Sep 04, 2014 10:38:32 AM com.bio4j.model.uniprot.programs.ImportUniprot importUniprot
SEVERE: com.ohnosequences.typedGraphs.TypedElementIndex$Unique.getElement(TypedElementIndex.java:38)
Sep 04, 2014 10:38:32 AM com.bio4j.model.uniprot.programs.ImportUniprot importUniprot
SEVERE: com.ohnosequences.typedGraphs.TypedVertexIndex$Unique.getVertex(TypedVertexIndex.java:41)
Sep 04, 2014 10:38:32 AM com.bio4j.model.uniprot.programs.ImportUniprot importUniprot
SEVERE: com.bio4j.model.uniprot.programs.ImportUniprot.importUniprot(ImportUniprot.java:337)
Sep 04, 2014 10:38:32 AM com.bio4j.model.uniprot.programs.ImportUniprot importUniprot
SEVERE: com.bio4j.titan.model.uniprot.programs.ImportUniprotTitan.execute(ImportUniprotTitan.java:56)
Sep 04, 2014 10:38:32 AM com.bio4j.model.uniprot.programs.ImportUniprot importUniprot
SEVERE: com.ohnosequences.util.ExecuteFromFile.main(ExecuteFromFile.java:66)
Sep 04, 2014 10:38:32 AM com.bio4j.model.uniprot.programs.ImportUniprot importUniprot
SEVERE: com.bio4j.titan.programs.ImportTitanDB.main(ImportTitanDB.java:11)

It seems to be some internal null value exception thrown in TitanTypedVertexIndex class.
@eparejatobes Any ideas why?

Consider adding locks to Transaction

Neo4j has entity-level locks in the scope of a transaction, which is a really useful feature. I think we could safely add this to the Transaction interface here, throwing UnsupportedOperation if missing from an implementation.