Git Product home page Git Product logo

gencode_regions's Introduction

gencode_regions

Extract 3'UTR, 5'UTR, CDS, Promoter, Genes from GTF files.

Data

If you only care about the final output, they are hosted build and GTF version wise on riboraptor.

Using Python

Dependencies

We recommend setting up a conda environment with Python>=3 and Python<=3.7 with gffutils v0.9 and pybedtools:

conda create --name gencode_env python=3.7
conda activate gencode_env
conda install -c bioconda gffutils=0.9 pybedtools

Notebooks

The corresponding output gzipped beds are in the data directory.

Using R

Dependencies

Run

./create_regions_from_gencode.R <path_to_GFF/GTF> <path_to_output_dir>

Will create exons.bed, 3UTR.bed, 5UTR.bed, genes.bed, cds.bed in <output_dir>

Example

  • Download GFF/GTF(GRCh37, v25, comprehensive, CHR) from gencodegenes.org:
   wget ftp://ftp.sanger.ac.uk/pub/gencode/Gencode_human/release_25/gencode.v25.annotation.gff3.gz \
   && gunzip gencode.v25.annotation.gff3.gz
  • Create regions:
./create_regions_from_gencode.R gencode.v25.annotation.gff3 /path/to/GRCh37/annotation

First exons, Last exons

We use GenePred format to make the process a bit simple.

  • Download gtfToGenePred

  • Convert gtf to GenePred:

    gtfToGenePred gencode.v25.annotation.gtf gencode.v25.annotation.genepred
    
  • Extract first exons:

    python genepred_to_bed.py --first_exon gencode.v25.annotation.genepred
    
  • Extract last exons:

    python genepred_to_bed.py --last_exon gencode.v25.annotation.genepred
    

Confused about exons and UTRs?

This should be helpful:

img

Source: Wikipedia

or probably this:

img

Source: Biostar

gencode_regions's People

Contributors

anamika-yadav99 avatar bug1303 avatar saketkc avatar tanishq67 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

gencode_regions's Issues

Not working with gencode.mm10.vM14.annotation.gtf / gff3

/mnts/stemcell-bioinfo/git/gencode_regions/create_regions_from_gencode.R /mnts/stemcell-bioinfo/genomes/gencode.mm10.vM14.annotation.gtf /mnts/stemcell-bioinfo/genomes/gencode_mm10.vM14
Import genomic features from the file as a GRanges object ... OK
Prepare the 'metadata' data frame ... OK
Make the TxDb object ... OK
[1] 0
Error in unlist_obj(introns.data) :
could not find function "elementNROWS"
Execution halted

I also checked this wasn't just a GTF issue by trying the same with the equivalent GFF file:

wget ftp://ftp.sanger.ac.uk/pub/gencode/Gencode_mouse/release_M14/gencode.vM14.annotation.gff3.gz
gunzip gencode.vM14.annotation.gff3.gz
mv gencode.vM14.annotation.gff3 /mnts/stemcell-bioinfo/genomes/.

/mnts/stemcell-bioinfo/git/gencode_regions/create_regions_from_gencode.R /mnts/stemcell-bioinfo/genomes/gencode.vM14.annotation.gff3 /mnts/stemcell-bioinfo/genomes/gencode_mm10.vM14
Import genomic features from the file as a GRanges object ... OK
Prepare the 'metadata' data frame ... OK
Make the TxDb object ... OK
[1] 0
Error in unlist_obj(introns.data) :
could not find function "elementNROWS"
Execution halted

can not execute gtfToGenePred

Hi:

I tried to use your code to get UTR and CDS from a gtf file from emsembl. For that, I download the gtfToGenePred file and do the following steps:
$ chmod +x gtfToGenePred` `conda install -c bioconda pybedtools conda install -c bioconda gffutils`

But when I typed
$ gtfToGenePred
-bash: gtfToGenePred: command not found

I am running under anaconda3 so the python version is >=3

What is the solution for the first step?

the position of first intron

I am a little bit confused about the first intron, just using the first one in the introns.bed of each gene or we should also use the *.genepred file and consider more things about it.

Thank you so much, your code is pretty valuable.
I am looking forward to your reply.

In your R script (create_regions_from_gencode.R) there is a small bug

transcripts.data <- unlist(transcripts.data) ###ERROR
exons.data <- unlist(exons.data)
threeUTRs.data <- unlist(threeUTRs.data)
fiveUTRs.data <- unlist(fiveUTRs.data)
cds.data <- unlist(cds.data)
introns.data <- unlist(introns.data)
genes.data <- unlist(genes.data) ####ERROR

There error is:

Error in getListElement(x, i, ...) : 
  GRanges objects don't support [[, as.list(), lapply(), or unlist() at
  the moment
Calls: unlist ... FUN -> [[ -> [[ -> getListElement -> getListElement
Execution halted

I did not need those lines to work for my purposes, but just an FYI on the issue.

Promoters?

Hi,

Are promoters also output specifically? It does not seem to be in the master data, nor does it say it is one of the outputs.
I understand that what makes a promoter is variable, but extracting xxx bp upstream/down of TSS would be good enough.

Thanks!

Issues faced while trying to run notebooks

I tried to run GRCg6.ipynb notebook in google colab first but I ran into many errors,
Later, after searching a lot I came to realize we need gffutils version 9.0 to run them.
After installing gffutils 9.0 I reran the notebook this time ran into new errors.
Then after spending some time on Stack overflow, I realized that we need Anaconda environment.
I installed Anaconda, then tried again and later figured out that pybedtools is not available for Windows ( only available for MacOS and Linux).
Then I installed the Anaconda in Linux operating system, and then realized that pybedtools doesn't support python 3.8 or 3.9, however I was able to install the library in python 3.7.

I re-ran the notebook and it worked perfectly :).

Conclusion:
We need anaconda environment with python <=3.7 and Linux/macOs operating system.
Also, we need gffutils version 0.9 ( corresponding command conda install -c bioconda gffutils=0.9 )

If you will allow me, then I can put up a PR adding these in the form of FAQ in the readME, which can help others.

extra information on the regions

The script looks great.

However, is it possible to get some extra information on the genomic features than just the coordinates ? For example, for the exons, would it be possible to know

  • which gene a particular exon belongs to
  • its position in the exon set - first, second ... last
    etc.

Thanks!

gencode 30 and ensembl 97 gtf/gff3 not working

I'm running the following command:
/home/zyh/software/gencode_regions/create_regions_from_gencode.R /home/zyh/ref/gencode.v30.annotation.gff3 /home/zyh/ref/gffutils/

and it gives error message:

Import genomic features from the file as a GRanges object ... OK
Prepare the 'metadata' data frame ... OK
Make the TxDb object ... OK
Warning messages:
1: In .get_cds_IDX(mcols0$type, mcols0$phase) :
  The "phase" metadata column contains non-NA values for features of type
  stop_codon. This information was ignored.
2: In .extract_transcripts_from_GRanges(tx_IDX, gr, mcols0$type, mcols0$ID,  :
  the transcript names ("tx_name" column in the TxDb object) imported
  from the "transcript_id" attribute are not unique
[1] 207646
Warning message:
In .set_group_names(ans, use.names, txdb, "tx") :
  some group names are NAs or duplicated
Warning message:
In .set_group_names(ans, use.names, txdb, "tx") :
  some group names are NAs or duplicated
Warning message:
In .set_group_names(grl, use.names, txdb, by) :
  some group names are NAs or duplicated
Warning message:
In .set_group_names(grl, use.names, txdb, by) :
  some group names are NAs or duplicated
Warning message:
In .set_group_names(ans, use.names, x, "tx") :
  some group names are NAs or duplicated
Error in h(simpleError(msg, call)) : 
  error in evaluating the argument 'obj' in selecting a method for function 'unname': GRanges objects don't support [[, as.list(), lapply(), or unlist() at
  the moment
Calls: unlist ... FUN -> [[ -> [[ -> getListElement -> getListElement
Execution halted

Using gtf as input caused another error:

Import genomic features from the file as a GRanges object ... OK
Prepare the 'metadata' data frame ... OK
Make the TxDb object ... OK
Warning message:
In .get_cds_IDX(mcols0$type, mcols0$phase) :
  The "phase" metadata column contains non-NA values for features of type
  stop_codon. This information was ignored.
[1] 0
Error in h(simpleError(msg, call)) : 
  error in evaluating the argument 'obj' in selecting a method for function 'unname': GRanges objects don't support [[, as.list(), lapply(), or unlist() at
  the moment
Calls: unlist ... FUN -> [[ -> [[ -> getListElement -> getListElement
Execution halted

Both original gtf and gff3 files downloaded from ensembl or gencode do not work. May I ask how to debug it?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.