geneontology / gocamgen Goto Github PK
View Code? Open in Web Editor NEWBase repo for constructing GO-CAM model RDF
Base repo for constructing GO-CAM model RDF
The gene MGI:1859682 is annotated to GO:0008520 with an IMP evidence code. The original annotation has an allele value in the with field (MGI:2180333). The 'with' field is missing from the model on noctua-dev.
Original annotation in GPAD:
MGI MGI:1859682 enables GO:0008520 MGI:MGI:2178849|PMID:11984597 ECO:0000315 MGI:MGI:2180333 20030212 MGI
The basic idea here is that when a relationship exists between terms in the ontology, e.g. MF part_of BP, then if there are manual annotations to both (and either no extensions or extensions are all the same), then those annotations could be connected during the import.
We need some illustrative examples here, so holding off on adding this to the to-do list just yet.
Create triples in model for only manual, non-PAINT annotations (experimental evidence codes, plus ISS) for now. Maybe turn the filter into an argument.
@thomaspd @vanaukenk Should the high-throughput codes (HTP, HDA, HMP, HGI, HEP) also be included in these models?
The code I have already written for AssocGoCamModel looks like it should take a list of ontobio associations and generate a single GO-CAM model, though I seem to have left AssocGoCamModel.translate() in an unfinished state:
>>> model.translate()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/ebertdu/go/gocamgen/gocamgen/gocamgen.py", line 257, in translate
aspect_triples.append(self.emit(term, ENABLED_BY, annoton.enabled_by))
AttributeError: 'AssocGoCamModel' object has no attribute 'emit'
Fixing this method and then model.write()
should get us a model we can load to Noctua for viewing.
In this spreadsheet we have all relations used in annotation extensions, at least in WB and MGI files. For many of these, I can easily map to an RO or BFO term:
regulates_transport_of -> RO:0002011
And the rest are available in the GOREL ontology (produced by the GO pipeline?), with some having xrefs to RO. Unfortunately I only see two of our unmapped relations having RO xrefs (has_agent
and regulates_level_of
). We'll need to figure out how to translate the other GORELs into GO-CAM if the GOREL relations aren't available for viewing in Noctua.
So far we've been "collapsing/consolidating," based on certain criteria (e.g. GP + term + extensions are same), multiple GPAD lines into distinct assertion individuals containing multiple evidence. I'd like to get clarification and document this criteria here first, then we can move it to the wiki page.
In my head, this is basically a header vs line situation so I'll present it like so:
Header:
Line:
@vanaukenk @ukemi @thomaspd This mainly came about recently when trying to figure out how to group lines by the with/from
field, hence the question mark. Do multiple GPAD lines that differ only in with/from
values represent the same assertion individual in GO-CAM or multiple?
For MOD imports, we also want to include annotations that use one of the 'high throughput evidence used in manual assertion' evidence codes. These are:
ECO:0006056 | High throughput evidence used in manual assertion | HTP
ECO:0007001 | High throughput mutant phenotype evidence used in manual assertion | HMP
ECO:0007003 | High throughput genetic interaction evidence used in manual assertion | HGI
ECO:0007005 | High throughput direct assay evidence used in manual assertion | HDA
ECO:0007007 | High throughput expression pattern evidence used in manual assertion | HEP
Thx.
Add step at end of model generation to validate model against the GO-CAM spec here in ShEx format:
https://raw.githubusercontent.com/geneontology/go-shapes/master/shapes/go-cam-shapes.shex
Would be cool if I could integrate some python ShEx parser like PyShEx to look directly at RdfLib graph object in memory. This way I can immediately respond to errors before writing out file. PyShEx
has a CLI we can fallback on (checking the written out .ttl
file) if using as a library doesn't work out.
First step of this ticket is to get some sort of feedback on models coming out. Then we'll look at some common errors/issues.
For @ukemi from our 2019-08-29 call. Here's what I have down:
I'll adjust formatted_ext_patterns.tsv accordingly in a new branch, regenerate a new bad_extensions.tsv
report, and I'll request PR approval from @ukemi for when he thinks the changes are good.
When annotation extensions are comma delimited and are from GO_CC, CL and an anatomy ontology then the model should indicate GO_CC<>part_of<>CL<>part_of<>anatomy.
MGI | MGI:1336882 | acts_upstream_of_or_within | GO:0070625 | MGI:MGI:3801544|PMID:18535671 | ECO:0000315 | MGI:MGI:3056083 | 20160406 | MGI | occurs_in(EMAPA:35651),occurs_in(CL:0002064),occurs_in(GO:1990794)
See the bottom of model:
http://noctua.geneontology.org/editor/graph/gomodel:5c4605cc00000315
Right now its just my ORCID and the GO group URL hard-coded in. Should be able to specify these at many levels, e.g. model, axiom, individual, evidence.
I'm creating some new annotations in Noctua that are just part of gene-by-gene curation, i.e. not part of a larger GO-CAM model.
I've named this model according to our import conventions, i.e. WB:WBGene00003480.
When we actually import the annotations into Noctua, it would be nice if we could merge other annotations to WBGene00003480 into that existing model.
Would that be possible?
Thx.
For genes annotated to root node for all aspects, BP, MF, and CC:
Create a single 'annoton' of:
GP-A<-enabled_by-[root MF]-part_of->[root BP]
and [root MF] -> occurs_in [root CC]
Looks like a heterogeneous doubling-up of prefix prepending for GO class(es?)
<http://identifiers.org/http://purl.obolibrary.org/obo/GO_0003674> a owl:Class .
At least for this MF root term in WB_WBGene00004055.ttl
For each statement that gets generated from a line of GPAD, add an OWL annotation containing a String capturing the full GPAD line. Something like:
rdfs:comment "GPAD_ORIGIN: WB WBGene00000816 part_of GO:0005634 GO_REF:0000037 ECO:0000501 UniProtKB-KW:KW-0539 20180707 UniProt "
This can be attached to the evidence block. It will be useful for debugging and can be removed once the importer is complete (or left as part of the provenance trail).
Loaded models generated from gocamgen somehow break the GPAD export function in minerva that drives the export in Noctua.
Last time GPAD export broke for my generated models it was with the SynGO models. The culprit there ended up being that I was using certain property that wasn't supported by minerva. I'll look up what the property was and see if it's in the gocamgen models.
Figure out where these NOT qualifiers go in the model. I'm guessing an axiom is involved. An example GPAD line from wb.gpad:
WB WBGene00000038 NOT|enables GO:0003990 PMID:10891266|WB_REF:WBPaper00004251 ECO:0000320 20170131 WB
Write a GAF-to-GPAD conversion step (just use ontobio's parser/writer) for whenever GPAD isn't available for a MOD and we have to use GAF.
Perhaps always write out the converted GPAD before subsequent conversion to GO-CAM.
This can be done relatively easily by calling uri() on the ontobio writer (TurtleRdfWriter). Recent ontobio update points to this new JSONLD. There may still be a few rdflib.term.URIRef() calls that explicitly set prefix (possibly incorrectly).
For context:
geneontology/go-site#617
biolink/ontobio#167
As part of the pipeline, we need to output a report that lists the lines in the GPAD file that do NOT get converted according to the current iteration of the rules so curators can check the annotation to see if it's correct (and the rule revised) or should be fixed.
For an annotation of Gene A to 'regulation of X', the models need to be decomposed into a regulatory process and the process that is being regulated. So if a gene product is annotated to 'regulation of leukocyte differentiation' the annoton would look like:
MF enabled_by A, MF-part_of;causally upstream of etc-generic BP, generic BP-regulates-X.
For annotation extensions, when the relation is a property chain like regulates_o_relation-T then the annotation extension should be added to the regulated process X. So the annotation extension would be X-relation-T.
MGI | MGI:108078 | acts_upstream_of_or_within | GO:0042662 | MGI:MGI:3711603|PMID:17462603 | ECO:0000316 | MGI:MGI:98956 | 20091104 | MGI | regulates_o_results_in_specification_of(CL:0000352) |
---|
Check for consistency.
Maybe also check with GO Rules ?
Gene products annotated to BP terms with the 'acts upstream of' or 'acts upstream of or within' relation should be imported as, for example:
GP <- enabled_by [root MF] -> causally_upstream_of BP
The equivalent model would be used for 'acts upstream of or within' except that the relation between the root MF and the BP would be 'causally upstream of or within'.
See http://noctua-dev.berkeleybop.org/editor/graph/gomodel:4187773d-e2b6-4235-895c-40436435d60d
for how they are currently imported.
The method input would be an ontobio AssociationSet or association objects. This would update the model object, which could then be written out later.
May need to port over annotation logic from go-pombase.
Translate whatever has_regulation_target
annotation extensions satisfy one of the scenarios described in the google doc.
@vanaukenk @ukemi @thomaspd A question about when the use of has_regulation_target
is valid: On the extensions2GO-CAM wiki, has_regulation_target
is stated as only valid when primary term is an MF. This conforms with my current rule-checking code. Meanwhile, the google doc above lists strategies for how to handle these when the primary term is one of the listed BP terms or its descendants e.g.
3.a. GP-A [regulation of molecular function Z] has_regulation_target GP-B
In searching the GPADs for examples I see has_regulation_target
used on annotations to both MF and BP terms:
MGI MGI:107771 enables GO:0005096 MGI:MGI:3698699|PMID:17116687 ECO:0000314 20101101 MGI has_regulation_target(MGI:MGI:97846)|has_regulation_target(MGI:MGI:2180784)
WB WBGene00000222 involved_in GO:0006990 PMID:16184190|WB_REF:WBPaper00026830 ECO:0000315 20131015 WB has_regulation_target(WB:WBGene00002783)
Can someone set me straight on this?
Collect examples of WB and MGI genes and their annotations for testing the import and GPAD generation pipelines. Annotations are in spreadsheets in the January Berkeley meeting folder: https://drive.google.com/drive/folders/1aMocjkYOUSd8rxWXCjGOUwYIu6Qnc-y7
Add ECO code filtering option to library like I did here.
To test that all of the information content in an input GPAD file is indeed captured in the set of GO-CAMs generated from that file, make a module to compare the input GPAD with the GPAD that minerva generates from the GO-CAM.
Close by providing a line by line report of whether each input gpad is recapitulated, deepened, or lost.
Provide model-level report about whether model is (OWL) consistent or not and a count of the input gpad lines in the categories above. e.g. model for Gene A, is consistent, has 4 recapitulations, 2 deepened, 2 lost, and 1 new.
I spotted this in a MGI model for two annotations to the same ID and term:
MGI:MGI:87859 to GO:0004713
I'm suspecting it's due to emitting evidence URI, dates, comments triples to an axiom for each new evidence rather than appending the evidence URI to the LEGO:evidence property of the single axiom. May the solution is similar to this?
[ rdf:type owl:Axiom ;
owl:annotatedSource <http://model.geneontology.org/66ab4fd6-68b2-4e3c-aa19-88199eca825e/86ed125d-a654-426d-aaa6-48a30f750a95> ;
owl:annotatedProperty <http://purl.obolibrary.org/obo/RO_0002333> ;
owl:annotatedTarget <http://model.geneontology.org/66ab4fd6-68b2-4e3c-aa19-88199eca825e/e46feaf1-2984-43d0-a3ed-d292cead6ef0> ;
<http://geneontology.org/lego/evidence> <http://model.geneontology.org/66ab4fd6-68b2-4e3c-aa19-88199eca825e/61d635f0-2397-4c0c-908a-740637d14b9d> , <http://model.geneontology.org/66ab4fd6-68b2-4e3c-aa19-88199eca825e/949d06a6-3355-4614-b8a3-2dbdef877c1f> ;
<http://purl.org/dc/elements/1.1/contributor> "http://orcid.org/0000-0002-6659-0416" ;
<http://purl.org/dc/elements/1.1/date> "2014-07-14" ;
rdfs:comment "MGI MGI:87859 enables GO:0004713 MGI:MGI:2384444|PMID:12167702 ECO:0000315 20140714 MGI has_direct_input(MGI:MGI:104741)"
] .
Specifically the line:
<http://geneontology.org/lego/evidence> <http://model.geneontology.org/66ab4fd6-68b2-4e3c-aa19-88199eca825e/61d635f0-2397-4c0c-908a-740637d14b9d> , <http://model.geneontology.org/66ab4fd6-68b2-4e3c-aa19-88199eca825e/949d06a6-3355-4614-b8a3-2dbdef877c1f> ;
Finish up pulling in these fields from GPAD:
Contributor should be orcID. In some lines (like protein2go) this is in the curator_uri
property in annotation properties column. If curator_uri
not present, xref name against users.yaml
to try getting orcID. If still can't find orcID, leave blank, one will need to be created.
In this model:
http://noctua-dev.berkeleybop.org/editor/graph/gomodel:a4631c87-4d56-4e4c-92cd-ede05da3483b
the inputs for the annotation:
mec-3 contributes_to -> RNA polymerase II regulatory region sequence-specific DNA binding (GO:0000977)
are folded into the mec-3 GP node, but should be folded into the MF node.
The reasoner turns red on this model.
Note also that there are three has_inputs, but the one input that doesn't render a gene name (just shows the URI) is actually a curation mistake and has been removed at the source.
From evidence rules 2 and 3 in the Google doc, we said that if there was more than one reference separated by a pipe, we would import only one reference using this descending list of preferences:
Order of preference for references:
PMID
GO_REF
doi
Other resource, e.g. MOD paper ID
Do we still want to do this?
Right now, it looks like both IDs (i.e. PMID and WBPaper ID) are being imported from the WB annotations, but only the WBPaper ID is displayed in the graph editor and both are exported in the Annotation Preview.
I can't quite make out how the paper ID is selected for the corresponding form view; both seem to be used but I can't discern a pattern.
Currently for MGI, we're filtering out all ISO annotation lines from GPAD. We now want to allow ISO lines to be translated in model if that ISO line has Assigned_by=MGI
.
This shouldn't affect other MOD filters. Yet.
From David's email:
The ones to be filtered would be ones with the references:
GO_REF:0000096, MGI: 4834177
GO_REF:0000096; MGI: 4417868
For importing the WB annotations, we will need to first filter the gpad file using the 'WB' value in the Assigned_by field.
In the current import on noctua-dev, it looks like we also have manual annotations from other groups, for example, Assigned_by 'UniProt'.
We have this in the Google doc but just to note an example here for testing purposes:
The inputs on the mec-3 contributes_to RNA polymerase II regulatory region, sequence-specific DNA binding are pipe-separated and should be split out into separate annotations.
This can happen when there are multiple evidence for the same GP-to-term.
Will need to find existing triple, if exists, matched by GP and term classes (and relation, once we get that fancy) and add the evidence from each identical annotation on that same triple.
A WB example to work with is WB:WBGene00000903 GO:0040024
, which has 8 lines in the current wb.gpad file.
Currently this method just appends input to http://identifiers.org/
. A JSONLD (like go_context.jsonld) should be plugged in somewhere and used for when CURIEs are declared.
E.g.
>>> model = GoCamModel("test", context_jsonld="go_context.jsonld")
>>> model.declare_class("PomBase:SPAC19D5.01")
"http://identifiers.org/pombase/SPAC19D5.01"
Hopefully this will help prevent identifier inconsistencies suppressing GP labels.
This ticket is to keep track of the efforts to refactor @dustine32 gpad-to-ttl code to use a reusable gocam data model.
The gocam data model specifications is currently under major discussions for further refinements. Once this is done and the python implementation of that model is completed, we can move forward into integrate it in this gpad-to-ttl code.
Deliverables:
Doing some QC on the imports, I realized that WB (and others) have annotations to other children of 'binding' (GO:0005488) that use the IPI (ECO:0000353) evidence code and contain an entry in the With/From field.
To be consistent in how we model 'binding' annotations in GO-CAM, we'll need to broaden our conversion rule so that any annotation to GO:0005488 that has an entry in the With/From field, will convert the With/From value to a 'has input' annotation extension, just like we're doing for 'protein binding' (GO:0005515) and its children.
For an example, see the 'miRNA binding' (GO:0035198) annotation in this alg-1 model.
@dustine32 - let me know if you need any more info or examples. Thx.
After much discussion amongst @pgaudet @thomaspd @ukemi @vanaukenk , we've agreed on a representation for modeling 'regulates o occurs in' when the primary GO annotation is to a [regulation of BP] term.
The representative example is the top line of this model:
http://noctua.geneontology.org/editor/graph/gomodel:5c4605cc00000396
For a conventional annotation of:
GP acts_upstream_of_or_within [regulation of BP] regulates_o_occurs_in CC
The rule could be written as:
GP <- enabled_by [root MF] -> causally_upstream_of_or_within [regulation of BP] -> regulates [BP] <- occurs_in [CC]
If the original regulation annotation was to a positive or negative regulation term, then we could substitute 'positively regulates' or 'negatively regulates' for 'regulates' in the above model.
See also: geneontology/go-annotation#2626
GO terms used in exported ttl
use an identifier.org uri like http://identifiers.org/GO:0000302
they should always be using purls like
http://purl.obolibrary.org/obo/GO_0000302
As part of the input code, we want to change annotation extensions that use 'has direct input' to 'has input'.
As suggested by @goodb .
Will change model ID generation code, which currently grabs a random UUID, to instead simply use the model gene's identifier (e.g. WB:WBGene00003167, MGI:MGI:2159711).
Example model ID: gomodel:WB:WBGene00003167
That extra colon might wreak some havoc so the actual model ID pattern could change. This will help with keeping static URLs to test models after multiple iterations of generating models.
WB:WBGene00006498 (ten-1) should have 17 Biological Process annotations associated with it, but right now, I only see the two CC annotations.
http://noctua-dev.berkeleybop.org/editor/graph/gomodel:e2ebe743-1704-4716-9a86-a9aa6ca7306f
Some examples of missing annotation lines from the snapshot GPAD:
WB WBGene00006498 involved_in GO:0007276 PMID:15936327|WB_REF:WBPaper00025239 ECO:0000315 20090108 WB
WB WBGene00006498 involved_in GO:0008406 PMID:18632986|WB_REF:WBPaper00032026 ECO:0000315 WB:WBVar00091925 20090108 WB
WB WBGene00006498 involved_in GO:0009792 PMID:18632986|WB_REF:WBPaper00032026 ECO:0000316 WB:WBGene00000961 20090108 WB
WB WBGene00006498 involved_in GO:0009792 PMID:18632986|WB_REF:WBPaper00032026 ECO:0000316 WB:WBGene00001328 20090108 WB
WB WBGene00006498 involved_in GO:0009792 PMID:18632986|WB_REF:WBPaper00032026 ECO:0000316 WB:WBGene00002081 20090108 WB
WB WBGene00006498 involved_in GO:0009792 PMID:18632986|WB_REF:WBPaper00032026 ECO:0000316 WB:WBGene00003738 20090108 WB
WB WBGene00006498 involved_in GO:0009792 PMID:22938866|WB_REF:WBPaper00041503 ECO:0000316 WB:WBGene00002251 20121022 WB
WB WBGene00006498 involved_in GO:0009792 PMID:22938866|WB_REF:WBPaper00041503 ECO:0000315 WB:WBVar00091925 20121022 WB
WB WBGene00006498 involved_in GO:0010172 PMID:21795395|WB_REF:WBPaper00039988 ECO:0000316 WB:WBGene00001077 20111103 WB
The extensions spreadsheet I've been producing using gpad_extensions_mapper.py
has been filtering the input GPAD lines using logic different from that used for filtering GPAD when translating into models. I should reuse the same FilterRule
-derived classes in gpad_extensions_mapper.py
.
Specifically, extensions generated by GO-CAM-exported GPAD are being included in the extensions spreadsheet.
Work in progress taking place in https://docs.google.com/document/d/1AgkJqA4-V0HG0SaDiNO9ENTxaQC_1HwsmwyhyHhuxog/edit#
We had initially decided to infer this GO-CAM structure from an annotation's aspect:
Since we're now using GPAD, where the relation is explicitly stated, we can just create triples using this instead (after converting to RO of course). So, for this example line (where involved_in
translates to RO:0002331
):
WB WBGene00000001 involved_in GO:0008286 PMID:12393910|WB_REF:WBPaper00005614 ECO:0000316 WB:WBGene00000898 20151214 WB
Triple will be WB:WBGene00000001-RO:0002331->GO:0008286
.
For a cellular component annotation, the model should just indicate GP<>part_of<>CC, not GP<>enables<>MF<>coccurs_in<>CC.
See model
gomodel:785e6d6b-fbda-4b41-9268-6493c8ee37a1
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.