Matrix-like representations of mutation and CN data
Home Page: https://bioconductor.org/packages/RaggedExperiment
Hi,
I have a total of 124 samples (each of them a GRanges objetct) in a list.
transfor[1]
$EPR000148
GRanges object with 57 ranges and 1 metadata column:
seqnames ranges strand | score
<Rle> <IRanges> <Rle> | <integer>
[1] chr1 [ 129285, 6314822] * | 1
[2] chr1 [ 12855876, 13802437] * | 1
[3] chr1 [ 16344494, 33282775] * | 1
[4] chr1 [117622286, 148951207] * | 1
[5] chr2 [ 90464785, 95963242] * | 1
... ... ... ... . ...
[53] chr21 [10907208, 14756481] * | 0
[54] chr21 [44837551, 47552385] * | 1
[55] chr22 [18304392, 18900750] * | 1
[56] chr22 [21045398, 21742598] * | 1
[57] chr22 [42523409, 42779699] * | 1
-------
seqinfo: 20 sequences from an unspecified genome; no seqlength
I get the following warning:
Warning message:
In `[<-`(`*tmp*`, i, value = <S4 object of class structure("GRanges", package = "GenomicRanges")>) :
implicit list embedding of S4 objects is deprecated
Performing the function RaggedExperiment works perfect.
>ragedtransfor<-RaggedExperiment(transfor, colData=colDat)
> ragedtransfor
class: RaggedExperiment
dim: 3828 124
assays(1): score
rownames: NULL
colnames(124): EPR000148 EPR002842 ... EPR985377 EPR990968
colData names(1): id
but when performing any of the Assayfunctions, I get the following:
sparseAssay(ragedtransfor)
Error in as(background, class(mcol)) :
no method or default for coercing “logical” to “factor”
I performed the debug option and it crashes in the following step:
debug: dimnames[[1]] <- as.character(.rowRanges(x))
Browse[2]>
debug: na <- as(background, class(mcol))
Browse[2]>
Error in as(background, class(mcol)) :
no method or default for coercing “logical” to “factor”
Any clue about what´s happening? Thanks in advance
The following should work
example("RaggedExperiment")
rownames(re3) <- rev(letters[1:4])
colnames(re3) <- c("sampA", "sampB")
I can't see any way to get at individual GRanges:
> example("RaggedExperiment")
> grl[[1]]
GRanges object with 2 ranges and 1 metadata column:
seqnames ranges strand | score
<Rle> <IRanges> <Rle> | <integer>
a chr1 [ 1, 10] - | 1
b chr1 [11, 18] + | 2
-------
seqinfo: 2 sequences from an unspecified genome; no seqlengths
> grl$sample1
GRanges object with 2 ranges and 1 metadata column:
seqnames ranges strand | score
<Rle> <IRanges> <Rle> | <integer>
a chr1 [ 1, 10] - | 1
b chr1 [11, 18] + | 2
-------
seqinfo: 2 sequences from an unspecified genome; no seqlengths
> re3[[1]]
Error in re3[[1]] : this S4 class is not subsettable
> re3$sample1
Error in re3$sample1 : $ operator not defined for this S4 class
>
Thanks so much for RaggedExperiment! They are truly amazing for multi-omic and genomics in general.
I'm encountering a new error that is only occurring in the Bioc-devel branch of Bioconductor packages when converting a GRangesList object to a RaggedExperiment.
A reproducible example can be shown using the existing Bioconductor release (3.9) and upcoming release (3.10 - currently devel).
Passing in 3.9
BiocManager::install("biobenkj/compartmap")
#should be version 1.65.7
library(compartmap)
library(minfi)
data(meth_array_450k_chr14)
getArrayABsignal(array.data.chr14, parallel=F, chr="chr14", bootstrap=F, genome="hg19", array.type="hm450")
Filtering to open sea CpG loci...
Converting to squeezed M-values.
Imputing missing values.
Dropping samples with >80% NAs.
Imputing missing data with kNN.
Cluster size 3384 broken into 2771 613
Cluster size 2771 broken into 1052 1719
Done cluster 1052
Cluster size 1719 broken into 476 1243
Done cluster 476
Done cluster 1243
Done cluster 1719
Done cluster 2771
Done cluster 613
Working on naive.1
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on rTreg.2
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on act_naive.3
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on naive.4
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on act_naive.5
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on act_rTreg.6
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on naive.7
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on rTreg.8
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on act_naive.9
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on act_rTreg.10
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on birth.11
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
> array_compartments
class: RaggedExperiment
dim: 968 11
assays(2): pc compartments
rownames: NULL
colnames(11): naive.1 rTreg.2 ... act_rTreg.10 birth.11
colData names(10): Sample_Name Sample_Well ... Basename filenames
The session info:
> sessionInfo()
R version 3.6.1 (2019-07-05)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Mojave 10.14.5
Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] parallel stats4 stats graphics grDevices utils datasets methods base
other attached packages:
[1] minfi_1.30.0 bumphunter_1.26.0 locfit_1.5-9.1
[4] iterators_1.0.12 foreach_1.4.7 compartmap_1.65.6
[7] bsseq_1.20.0 BiocSingular_1.0.0 BSgenome.Mmusculus.UCSC.mm9_1.4.0
[10] Mus.musculus_1.3.1 TxDb.Mmusculus.UCSC.mm10.knownGene_3.4.7 org.Mm.eg.db_3.8.2
[13] BSgenome.Hsapiens.UCSC.hg38_1.4.1 BSgenome_1.52.0 rtracklayer_1.44.3
[16] Biostrings_2.52.0 XVector_0.24.0 Homo.sapiens_1.3.1
[19] TxDb.Hsapiens.UCSC.hg19.knownGene_3.2.2 org.Hs.eg.db_3.8.2 GO.db_3.8.2
[22] OrganismDbi_1.26.0 GenomicFeatures_1.36.4 AnnotationDbi_1.46.1
[25] RaggedExperiment_1.8.0 SummarizedExperiment_1.14.1 DelayedArray_0.10.0
[28] BiocParallel_1.18.1 matrixStats_0.54.0 Biobase_2.44.0
[31] GenomicRanges_1.36.0 GenomeInfoDb_1.20.0 IRanges_2.18.2
[34] S4Vectors_0.22.0 BiocGenerics_0.30.0
loaded via a namespace (and not attached):
[1] backports_1.1.4 plyr_1.8.4 igraph_1.2.4.1 lazyeval_0.2.2 splines_3.6.1
[6] ggplot2_3.2.1 digest_0.6.20 viridis_0.5.1 magrittr_1.5 memoise_1.1.0
[11] limma_3.40.6 readr_1.3.1 annotate_1.62.0 R.utils_2.9.0 askpass_1.1
[16] siggenes_1.58.0 prettyunits_1.0.2 colorspace_1.4-1 blob_1.2.0 dplyr_0.8.3
[21] crayon_1.3.4 RCurl_1.95-4.12 graph_1.62.0 genefilter_1.66.0 GEOquery_2.52.0
[26] zeallot_0.1.0 impute_1.58.0 survival_2.44-1.1 glue_1.3.1 registry_0.5-1
[31] gtable_0.3.0 zlibbioc_1.30.0 Rhdf5lib_1.6.0 HDF5Array_1.12.2 scales_1.0.0
[36] DBI_1.0.0 rngtools_1.4 bibtex_0.4.2 Rcpp_1.0.2 viridisLite_0.3.0
[41] xtable_1.8-4 progress_1.2.2 bit_1.1-14 rsvd_1.0.2 mclust_5.4.5
[46] preprocessCore_1.46.0 httr_1.4.1 RColorBrewer_1.1-2 pkgconfig_2.0.2 reshape_0.8.8
[51] XML_3.98-1.20 R.methodsS3_1.7.1 tidyselect_0.2.5 rlang_0.4.0 munsell_0.5.0
[56] tools_3.6.1 RSQLite_2.1.2 stringr_1.4.0 bit64_0.9-7 beanplot_1.2
[61] scrime_1.3.5 purrr_0.3.2 RANN_2.6.1 nlme_3.1-141 pbapply_1.4-2
[66] RBGL_1.60.0 doRNG_1.7.1 nor1mix_1.3-0 R.oo_1.22.0 xml2_1.2.2
[71] biomaRt_2.40.4 compiler_3.6.1 rstudioapi_0.10 tibble_2.1.3 stringi_1.4.3
[76] lattice_0.20-38 Matrix_1.2-17 permute_0.9-5 multtest_2.40.0 vctrs_0.2.0
[81] pillar_1.4.2 BiocManager_1.30.4 data.table_1.12.2 bitops_1.0-6 irlba_2.3.3
[86] R6_2.4.0 gridExtra_2.3 codetools_0.2-16 MASS_7.3-51.4 gtools_3.8.1
[91] assertthat_0.2.1 rhdf5_2.28.0 openssl_1.4.1 pkgmaker_0.27 withr_2.1.2
[96] GenomicAlignments_1.20.1 Rsamtools_2.0.0 GenomeInfoDbData_1.2.1 hms_0.5.1 quadprog_1.5-7
[101] grid_3.6.1 tidyr_0.8.3 base64_2.0 DelayedMatrixStats_1.6.0 illuminaio_0.26.0
The failing version for 3.10
Filtering to open sea CpG loci...
Converting to squeezed M-values.
Imputing missing values.
Dropping samples with >80% NAs.
Imputing missing data with kNN.
Cluster size 3384 broken into 2771 613
Cluster size 2771 broken into 1052 1719
Done cluster 1052
Cluster size 1719 broken into 476 1243
Done cluster 476
Done cluster 1243
Done cluster 1719
Done cluster 2771
Done cluster 613
Working on naive.1
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on rTreg.2
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on act_naive.3
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on naive.4
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on act_naive.5
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on act_rTreg.6
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on naive.7
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on rTreg.8
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on act_naive.9
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on act_rTreg.10
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Working on birth.11
Computing compartments for chr14
Calculating correlations...
Done...
Calculating eigenvectors.
Smoothing eigenvector.
Done smoothing.
Error in as(objects[[1L]], "CompressedGRangesList", strict = FALSE) :
no method or default for coercing “SimpleGRangesList” to “CompressedGRangesList”
The session info:
> sessionInfo()
R version 3.6.1 (2019-07-05)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.3 LTS
Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] parallel stats4 stats graphics grDevices utils datasets
[8] methods base
other attached packages:
[1] minfi_1.31.1
[2] bumphunter_1.27.0
[3] locfit_1.5-9.1
[4] iterators_1.0.12
[5] foreach_1.4.7
[6] compartmap_1.65.7
[7] bsseq_1.21.1
[8] BiocSingular_1.1.7
[9] BSgenome.Mmusculus.UCSC.mm9_1.4.0
[10] Mus.musculus_1.3.1
[11] TxDb.Mmusculus.UCSC.mm10.knownGene_3.4.7
[12] org.Mm.eg.db_3.8.2
[13] BSgenome.Hsapiens.UCSC.hg38_1.4.1
[14] BSgenome_1.53.2
[15] rtracklayer_1.45.6
[16] Biostrings_2.53.2
[17] XVector_0.25.0
[18] Homo.sapiens_1.3.1
[19] TxDb.Hsapiens.UCSC.hg19.knownGene_3.2.2
[20] org.Hs.eg.db_3.8.2
[21] GO.db_3.8.2
[22] OrganismDbi_1.27.1
[23] GenomicFeatures_1.37.4
[24] AnnotationDbi_1.47.1
[25] RaggedExperiment_1.9.1
[26] SummarizedExperiment_1.15.9
[27] DelayedArray_0.11.7
[28] BiocParallel_1.19.3
[29] matrixStats_0.55.0
[30] Biobase_2.45.1
[31] GenomicRanges_1.37.16
[32] GenomeInfoDb_1.21.2
[33] IRanges_2.19.16
[34] S4Vectors_0.23.25
[35] BiocGenerics_0.31.6
loaded via a namespace (and not attached):
[1] backports_1.1.5 BiocFileCache_1.9.1 plyr_1.8.4
[4] igraph_1.2.4.1 lazyeval_0.2.2 splines_3.6.1
[7] ggplot2_3.2.1 digest_0.6.21 viridis_0.5.1
[10] magrittr_1.5 memoise_1.1.0 limma_3.41.17
[13] readr_1.3.1 annotate_1.63.0 R.utils_2.9.0
[16] askpass_1.1 siggenes_1.59.0 prettyunits_1.0.2
[19] colorspace_1.4-1 blob_1.2.0 rappdirs_0.3.1
[22] dplyr_0.8.3 crayon_1.3.4 RCurl_1.95-4.12
[25] graph_1.63.0 genefilter_1.67.1 GEOquery_2.53.0
[28] zeallot_0.1.0 impute_1.59.0 survival_2.44-1.1
[31] glue_1.3.1 registry_0.5-1 gtable_0.3.0
[34] zlibbioc_1.31.0 Rhdf5lib_1.7.5 HDF5Array_1.13.9
[37] scales_1.0.0 DBI_1.0.0 rngtools_1.4
[40] bibtex_0.4.2 Rcpp_1.0.2 viridisLite_0.3.0
[43] xtable_1.8-4 progress_1.2.2 bit_1.1-14
[46] rsvd_1.0.2 mclust_5.4.5 preprocessCore_1.47.1
[49] httr_1.4.1 RColorBrewer_1.1-2 pkgconfig_2.0.3
[52] reshape_0.8.8 XML_3.98-1.20 R.methodsS3_1.7.1
[55] dbplyr_1.4.2 tidyselect_0.2.5 rlang_0.4.0
[58] munsell_0.5.0 tools_3.6.1 RSQLite_2.1.2
[61] stringr_1.4.0 bit64_0.9-7 beanplot_1.2
[64] scrime_1.3.5 purrr_0.3.2 RANN_2.6.1
[67] nlme_3.1-141 pbapply_1.4-2 RBGL_1.61.0
[70] doRNG_1.7.1 nor1mix_1.3-0 R.oo_1.22.0
[73] xml2_1.2.2 biomaRt_2.41.9 compiler_3.6.1
[76] curl_4.2 tibble_2.1.3 stringi_1.4.3
[79] lattice_0.20-38 Matrix_1.2-17 permute_0.9-5
[82] multtest_2.41.0 vctrs_0.2.0 lifecycle_0.1.0
[85] pillar_1.4.2 BiocManager_1.30.4 data.table_1.12.4
[88] bitops_1.0-6 irlba_2.3.3 R6_2.4.0
[91] gridExtra_2.3 codetools_0.2-16 MASS_7.3-51.4
[94] gtools_3.8.1 assertthat_0.2.1 rhdf5_2.29.3
[97] openssl_1.4.1 pkgmaker_0.27 withr_2.1.2
[100] GenomicAlignments_1.21.7 Rsamtools_2.1.6 GenomeInfoDbData_1.2.1
[103] hms_0.5.1 quadprog_1.5-7 grid_3.6.1
[106] tidyr_1.0.0 base64_2.0 DelayedMatrixStats_1.7.2
[109] illuminaio_0.27.1
The code in question here takes a list object that is output from lapply, converts it to a GRangesList object using:
as(my_list, "GRangesList")
and converts to a RaggedExperiment, where it fails (or passes).
Any idea what might have changed or is the issue?
Thanks so much!
Hi, I'm trying to use this class for an ATAC-seq single-cell experiment, which means that I have a (sparse)count matrix and a list of regions.
Here is an example of the regions I have
GRanges object with 111857 ranges and 4 metadata columns:
seqnames ranges strand | count percentile
<Rle> <IRanges> <Rle> | <numeric> <numeric>
[1] chr1 9790-10676 * | 242 0.3323529
[2] chr1 180654-181318 * | 220 0.3004193
[3] chr1 191155-192066 * | 139 0.1566107
[4] chr1 267573-268458 * | 357 0.4609814
[5] chr1 270881-271760 * | 104 0.0855557
And when I build the RaggedExperiment, I obtain these two assays, that are coming from the elementMetadata of the GRange, which I'm not interested in, because they are just metadata
> ragexp
class: RaggedExperiment
dim: 111857 1
assays(2): count percentile
rownames: NULL
colnames: NULL
colData names(0):
So, I'm not sure I'm rightly understanding the assays of this class, because there are no dedicated examples or a vignette dedicated section.
But I would expect to collect one or more count matrices through the classic assays=List(counts=myMatrix, sparseCounts=mydgCMatrix) in the object constructor.
I've also tried to do an assay assignment but obtaining the following result
> assay(ragexp, withDimnames=FALSE)=seu@assays$ATAC@counts
> ragexp
class: RaggedExperiment
dim: 111857 1
Error in (function (classes, fdef, mtable) :
unable to find an inherited method for function ‘mcols’ for signature ‘"dgCMatrix"’
So, what I'm saying is that (in case this class is thought to be used for single-cell data) maybe would be preferable to have a more classic approach for the construction of the object and support for sparse matrices.
Thanks for any clarification :)
Real data sources to use:
Nice examples:
This class is defunct in release and non-existent in devel versions of Bioconductor
Following up on previous comments (#17):
RaggedExperiment nicely pretends to be a SummarizedExperiment with regard to assays, rowRanges, and colData.
However, RaggedExperiment does not seem to support a concept such as rowData for annotation to its rowRanges.
Essentially when you do mcols<- or rowData<-, you're adding to "number of assays". Since each column in mcols is an "assay".
I wonder whether this can be modified.
When constructing a RaggedExperiment from a GRangesList, would it be possible to distinguish mcols that become assays and mcols that are annotation?
(Currently this turns even non-numeric mcols into assays)
General motivation for being able to annotate additional row information is as for a SummarizedExperiment - however, in my specific use case I'm working with CNV calls where I only want the copy number to become an assay, while I want to keep additional columns as annotation to each individual CNV call (such as probe identifiers, quality measures, etc).
From Vince @vjcitn:
Just wondering whether you would consider returning a GRanges
for the *Assay methods, perhaps optionally? I will be parsing the
rownames to create this for compactAssay. Probably completely
trivial for sparseAssay as the RaggedExperiment GRanges is
immediately available.
cc: @mtmorgan
It would be nice to also support annotation of colData after creation of the object
(as SummarizedExperiment does).
Currently, "colData<-" is not supported as e.g.
colData(ra)$myCol <- rep("x", ncol(ra))
gives
Error: unable to find an inherited method for function ‘colData<-’ for signature ‘"RaggedExperiment", "DataFrame"’
I think there is a small bug in qreduceAssay concerning the simplifyReduce
argument.
Following the corresponding Arguments section in ?qreduceAssay, the simplifyReduce
argument is a function / functional accepting arguments score, range, and qrange.
Where qrange
is a
GRangesinstance with the same length asscore, providing the query range window to which the corresponding scores apply
However, when following through the code of qreduceAssay:
> qreduceAssay
function (x, query, simplifyReduce, i = 1, withDimnames = TRUE,
background = NA_integer_)
{
1 if (missing(i) && ncol(.mcols(x)) == 0)
2 return(matrix(NA, 0, 0))
3 stopifnot_simplify_ok(simplifyReduce, 3L)
4 i <- .assay_i(x, i)
5 mcol <- .mcols(x)[[i]][.rowidx(x)]
6 dim <- .dim(x)
7 subject <- unname(rowRanges(x))
8 query <- granges(query)
9 olap <- findOverlaps(query, subject)
10 sidx <- subjectHits(olap)
11 row <- queryHits(olap)
12 col <- rep(seq_len(dim[[2]]), lengths(.assays(x)))[.rowidx(x)][sidx]
13 score <- mcol[sidx]
14 subject <- subject[sidx]
15 qranges <- query[row]
16 ranges <- restrict(subject, start = pmax(start(qranges),
17 start(subject)), end = pmin(end(qranges), end(subject)))
18 group <- (row - 1L) * max(col, 0) + col
19 group <- match(group, unique(group))
20 result <- simplifyReduce(unname(splitAsList(score, group)),
21 unname(splitAsList(ranges, group)), unname(qranges))
22 group <- !duplicated(group)
23 na <- as(background, class(result))
24 dimnames <- list(NULL, NULL)
25 if (withDimnames)
26 dimnames <- list(as.character(query), .dimnames(x)[[2]])
27 m <- matrix(na, nrow = length(query), ncol = dim[[2]], dimnames = dimnames)
28 idx <- cbind(row = row[group], col = col[group])
29 m[idx] <- result
30 m[, .colidx(x), drop = FALSE]
}
using the example RaggedExperiment from ?qreduceAssay
x <- RaggedExperiment(GRangesList(
GRanges(c(A = "chr1:1-10:-", B = "chr1:8-14:-", C = "chr2:15-18:+"),
score = 3:5),
GRanges(c(D = "chr1:1-10:-", E = "chr2:11-18:+"), score = 1:2)
), colData = DataFrame(id = 1:2))
query <- GRanges(c("chr1:1-14:-", "chr2:11-18:+"))
and plug that into lines 1-19 of the above code of qreduceAssay, you end up (in
line 20) with the three arguments of simplifyReduce be set to:
> score <- unname(splitAsList(score, group))
> score
IntegerList of length 4
[[1]] 3 4
[[2]] 1
[[3]] 5
[[4]] 2
> range <- unname(splitAsList(ranges, group))
> range
GRangesList object of length 4:
[[1]]
GRanges object with 2 ranges and 0 metadata columns:
seqnames ranges strand
<Rle> <IRanges> <Rle>
[1] chr1 1-10 -
[2] chr1 8-14 -
[[2]]
GRanges object with 1 range and 0 metadata columns:
seqnames ranges strand
[1] chr1 1-10 -
[[3]]
GRanges object with 1 range and 0 metadata columns:
seqnames ranges strand
[1] chr2 15-18 +
...
<1 more element>
-------
seqinfo: 2 sequences from an unspecified genome; no seqlengths
> qrange <- unname(qranges)
> qrange
GRanges object with 5 ranges and 0 metadata columns:
seqnames ranges strand
<Rle> <IRanges> <Rle>
[1] chr1 1-14 -
[2] chr1 1-14 -
[3] chr1 1-14 -
[4] chr2 11-18 +
[5] chr2 11-18 +
-------
seqinfo: 2 sequences from an unspecified genome; no seqlengths
and thus with
> length(score) == length(qrange)
[1] FALSE
which is inconsistent with the description above
qrangeis aGRangesinstance with the same length asscore
which is the actual desired behavior.
Closer inspecting this case, you find that by collecting all queryHits in
line 11 and 15 above, qranges contains also duplicated query hits within a
group of subject ranges belonging to the same query range.
These groups are defined in group, line 19:
> group
[1] 1 1 2 3 4
A fix could thus look like this:
> qreduceAssay
function (x, query, simplifyReduce, i = 1, withDimnames = TRUE,
background = NA_integer_)
{
1 if (missing(i) && ncol(.mcols(x)) == 0)
2 return(matrix(NA, 0, 0))
3 stopifnot_simplify_ok(simplifyReduce, 3L)
4 i <- .assay_i(x, i)
5 mcol <- .mcols(x)[[i]][.rowidx(x)]
6 dim <- .dim(x)
7 subject <- unname(rowRanges(x))
8 query <- granges(query)
9 olap <- findOverlaps(query, subject)
10 sidx <- subjectHits(olap)
11 row <- queryHits(olap)
12 col <- rep(seq_len(dim[[2]]), lengths(.assays(x)))[.rowidx(x)][sidx]
13 score <- mcol[sidx]
14 subject <- subject[sidx]
15 qranges <- query[row]
16 ranges <- restrict(subject, start = pmax(start(qranges),
17 start(subject)), end = pmin(end(qranges), end(subject)))
18 group <- (row - 1L) * max(col, 0) + col
19 group <- match(group, unique(group))
## fix ##
ugroup <- !duplicated(group)
result <- simplifyReduce(unname(splitAsList(score, group)),
unname(splitAsList(ranges, group)), unname(qranges)[ugroup])
group <- ugroup
##
23 na <- as(background, class(result))
24 dimnames <- list(NULL, NULL)
25 if (withDimnames)
26 dimnames <- list(as.character(query), .dimnames(x)[[2]])
27 m <- matrix(na, nrow = length(query), ncol = dim[[2]], dimnames = dimnames)
28 idx <- cbind(row = row[group], col = col[group])
29 m[idx] <- result
30 m[, .colidx(x), drop = FALSE]
}
I can open a PR if needed.
qreduceAssay: simplifyReduce
disjoinAssay: simplifyDisjoin
(In CNV analysis, I assume also for similar data types such as somatic mutation), it is often of interest to summarize invidual calls across the
population, (i.e. define CNV regions), for subsequent association analysis with e.g. phenotype data.
https://www.ncbi.nlm.nih.gov/pubmed/19812545
https://www.ncbi.nlm.nih.gov/pubmed/22539667 (Figure 1 for a sketch of the concept)
In the simplest case, this just merges overlapping individual calls into summarized regions.
However, this typically inflates CNVR size and trimming low-density areas
(usually <10% of the total contributing individual calls within a summarized region)
is advisable.
Alternatively, a threshold for reciprocal overlap between any two individual calls
can also be applied.
That means such a function would take a RaggedExperiment as Input and would return
a GRanges object storing the summarized genomic regions.
grl <- GRangesList(
sample1 = GRanges( c("chr1:1-10", "chr2:15-18", "chr2:25-34") ),
sample2 = GRanges( c("chr1:1-10", "chr2:11-18" , "chr2:25-36") ),
sample3 = GRanges( c("chr1:2-11", "chr2:14-18", "chr2:26-36") ),
sample4 = GRanges( c("chr1:1-12", "chr2:18-35" ) ),
sample5 = GRanges( c("chr1:1-12", "chr2:11-17" , "chr2:26-34") ) ,
sample6 = GRanges( c("chr1:1-12", "chr2:12-18" , "chr2:25-35") )
)
re <- RaggedExperiment(grl)
populationRanges(re, density=0.1)
GRanges object with 3 ranges and 0 metadata columns:
seqnames ranges strand
[1] chr1 [ 1, 12] *
[2] chr2 [11, 18] *
[3] chr2 [25, 36] *
populationRanges(re, density=0)
GRanges object with 2 ranges and 0 metadata columns:
seqnames ranges strand
[1] chr1 [ 1, 12] *
[2] chr2 [11, 36] *
populationRanges(re, density=1)
GRanges object with 13 ranges and 0 metadata columns:
seqnames ranges strand
[1] chr1 [ 1, 1] *
[2] chr1 [ 2, 10] *
[3] chr1 [11, 11] *
[4] chr1 [12, 12] *
[5] chr2 [11, 13] *
... ... ... ...
[9] chr2 [19, 24] *
[10] chr2 [25, 25] *
[11] chr2 [26, 34] *
[12] chr2 [35, 35] *
[13] chr2 [36, 36] *
Where as merging and disjoining are suited for the ideal case of matching calls between individuals, technical and biological variation typically results in data with all kinds of overlaps and fuzzy region borders. That would be a way to account for that.
Hi, I have a RaggedExperiment and I want to change its colnames(x) or row names of its colData(x) to a different sample id from its colData, and found that the dimnames() setter was not working as it claimed from the documentation.
Example RaggedExperiment
library(RaggedExperiment)
sample1 <- GRanges(c(featureA = "chr1:1-10:-",
featureB = "chr1:8-14:+",
featureC = "chr2:15-18:+"),
score = 3:5,
cpnum = 6:8)
sample2 <- GRanges(c(featureD = "chr1:1-10:-", featureE = "chr2:11-18:+"),
score = 1:2,
cpnum = 9:10)
colDat <- DataFrame(Sample_id = paste0("sample", 1:2),
New_id = paste0("sample_and_patient", 1:2),
Tissue = c("Lung", "Liver"),
Condition = c("Turmo", "Normal"))
# Assemble RaggedExperiment object
ragexp <- RaggedExperiment(sample1 = sample1,
sample2 = sample2,
colData = colDat)
# Add metadata
metadata(ragexp) <- list(genome_build = "hg19")
ragexp
Currently the colnames are sample1, sample2, but I want to change to the New_id from colDat. I was trying the dimnames and fount that
# getter works as expected
ragexp_dimnames <- dimnames(ragexp)
# try update the dimnames
ragexp_dimnames[[2]] <- colDat$New_id
# reset dimnames not working
dimnames(ragexp) <- ragexp_dimnames
The vignette, Section 6.4 qreduceAssay currently says:
First we define our summary function that calculates a weighted average score per query range. Note that there are three arguments to this function.
weightedmean <- function(scores, ranges, qranges)
sum(scores * width(ranges)) / sum(width(ranges))
However, the given example only uses two arguments in its body (scores and ranges).
Is this intended? How would an example using all three arguments look like?
As far as I understand the procedure underlying qreduceAssay, it first identifies overlapping rowRanges for each given query range and then accordingly summarizes across these ranges by applying simplifyReduce.
Thus, simplifyReduce actually only needs to know about the ranges and associated scores it summarizes over, doesn't it?
Hi Martin, @mtmorgan
In the example:
library(RaggedExperiment)
example("RaggedExperiment")
length(re3)
The length returned is 1. In SummarizedExperiment, length returns the number of rows (odd?).
Should this output be accepted for RaggedExperiment?
Considering this to be a matrix representation, length should return something like dim[[1]] * dim[[2]]?
Marcel
RaggedExperiment continues to rule for all our 'omics related work! I did notice something interesting yesterday when running compactSummarizedExperiment(), when I attempt to access the names of the assays in a large RE
# RaggedExperiment in question
> aaml
class: RaggedExperiment
dim: 36019710 1401
assays(2): pc compartments
rownames: NULL
colnames(1401): 813584_Dx 814465_Dx ... RO02776B RO02815
colData names(99): Timepoint Gender ... MLLT10 KMT2A
#size
> object_size(aaml)
1.01 GB
#names access
#high memory usage (100s of GB)
names(assay(aaml))
#names access
#near instant
assayNames(aaml)
it will either be near instantaneous with using assayNames(), or require 100s of GB of memory with names(assay(my_RE)). Do you know why this might be the case? I'll work on getting a smaller reproducible example if there is interest.
Thanks again for all that you do and RaggedExperiments!
Is this an expected error (RaggedExperiment_1.1.1 and 1.1.2)?
> qreduceAssay(ra, query = gistic,
+ simplify = mean,
+ i = "score",
+ background = 0)
Each of the 2 combined objects has sequence levels not in the other:
- in 'x': X
- in 'y': 9, 13, 16, 21, 23
Make sure to always combine/compare objects based on the same reference
genome (use suppressWarnings() to suppress this warning).Error: isTRUEorFALSE(na.rm) is not TRUE
>
Since GH doesn't allow .rda: re.rda.zip
I've run into some trouble working with a RaggedExperiment I created. I am seeing extra columns in data after it is sliced, and accessing the data only returns the unexpected columns. I have replicated the problem in one of the Waldron datasets http://s3.amazonaws.com/multiassayexperiments/ccleMAEO.rds
library(MultiAssayExperiment)
library(RaggedExperiment)
ccle <- updateObject(readRDS('ccleMAEO.rds'))
ccle <- ccle[,ccle$TissueOrigin == 'BREAST']
muts <- ccle[c('BRCA1','BRCA2'), ,'Mutations']@ExperimentList$Mutations
muts
class: RaggedExperiment
dim: 2 26
assays(0):
rownames(2): BRCA1 BRCA2
colnames(26): AU565_BREAST BT20_BREAST ... ZR751_BREAST ZR7530_BREAST
colData names(0):
The dimensions and names from this view are what we expect.
dimnames(muts)[2]
[[1]]
[1] "AU565_BREAST" "BT20_BREAST" "BT474_BREAST" "BT549_BREAST" "CAL851_BREAST" "CAMA1_BREAST" "EFM19_BREAST" "HCC1187_BREAST" "HCC1395_BREAST"
[10] "HCC1569_BREAST" "HCC1806_BREAST" "HCC1954_BREAST" "HCC70_BREAST" "HMC18_BREAST" "HS578T_BREAST" "HS739T_BREAST" "MCF7_BREAST" "MDAMB415_BREAST"
[19] "MDAMB436_BREAST" "MDAMB453_BREAST" "MDAMB468_BREAST" "SKBR3_BREAST" "T47D_BREAST" "UACC812_BREAST" "ZR751_BREAST" "ZR7530_BREAST"
However if we access the names of the assays in the RaggedExperiment, we have do not see what we expect after our earlier slicing, and we have a list of a different length than the previous dimnames() access. Notice there is now PROSTATE and LUNG despite having been sliced earlier on BREAST and the previous command only accessing BREAST.
names(muts@assays)
[1] "22RV1_PROSTATE" "AU565_BREAST" "BT20_BREAST" "BT474_BREAST" "BT549_BREAST" "CAL851_BREAST" "CALU3_LUNG" "CAMA1_BREAST" "EFM19_BREAST"
[10] "HCC1187_BREAST" "HCC1395_BREAST" "HCC1569_BREAST" "HCC1806_BREAST" "HCC1954_BREAST" "HCC70_BREAST" "HMC18_BREAST" "HS578T_BREAST" "HS739T_BREAST"
[19] "MCF7_BREAST" "MDAMB415_BREAST" "MDAMB436_BREAST" "MDAMB453_BREAST" "MDAMB468_BREAST" "SKBR3_BREAST" "T47D_BREAST" "UACC812_BREAST" "ZR751_BREAST"
[28] "ZR7530_BREAST"
Furthermore the unlist() function executed on the assays only returns the unexpected result, and not all the expected assays. We expect only BREAST and should see more entries. Instead we see PROSTATE and LUNG.
unlist(muts@assays)
GRanges object with 2 ranges and 0 metadata columns:
seqnames ranges strand
22RV1_PROSTATE.BRCA2 chr13 [32954022, 32954023] +
CALU3_LUNG.BRCA1 chr17 [41245233, 41245233] +
seqinfo: 23 sequences from hg19 genome; no seqlengths
Additionally the sparseAssay function described in the documentation did not seem to work on the RaggedExperiment.
sparseAssay(muts)
Error in .assay_i(x, i) : 'length(assays(x))' is 0
For more information about the session and versions:
sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.6
Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] parallel stats4 stats graphics grDevices utils datasets methods base
other attached packages:
[1] Biobase_2.37.2 RaggedExperiment_1.1.4 GenomicRanges_1.29.12 GenomeInfoDb_1.13.4 IRanges_2.11.12
[6] S4Vectors_0.15.6 BiocGenerics_0.23.0 MultiAssayExperiment_1.3.31
I was surprised that qreduceSummarizedExperiment has a different argument (simplify) than the corresponding argument of qreduceAssay (simplifyReduce). Shouldn't these functions have consistent argument names?
Same example as #21 (comment)
I would now like to change the order of the assays, particularly to make the Segment_mean (= copy number estimate) the default assay, instead of Num_probes (= number of SNPs used to infer the CN).
However, these attempts fail:
> assays(gbm.ra) <- assays(gbm.ra)[2:1]
Error in (function (classes, fdef, mtable) :
unable to find an inherited method for function ‘assays<-’ for signature ‘"RaggedExperiment", "SimpleList"’
> assays(gbm.ra, 1) <- assays(gbm.ra, 2)
Error in (function (classes, fdef, mtable) :
unable to find an inherited method for function ‘assays<-’ for signature ‘"RaggedExperiment", "SimpleList"’
> assay(gbm.ra) <- assays(gbm.ra)[[2]]
Error in (function (classes, fdef, mtable) :
unable to find an inherited method for function ‘assay<-’ for signature ‘"RaggedExperiment", "missing"’
The man page of "RaggedExperiment-package" says
"A distinction between the GRangesList and the RaggedExperiment classes is that
the RaggedExperiment class allows for ragged ranges, meaning that there may be
a different number of ranges for each sample."
This should be apparently:
"A distinction between the SummarizedExperiment and the RaggedExperiment classes is ..."
(as GRangesList also allows for a different number of ranges for each sample (where each sample is an element of the list))
LGeistlinger:
(2) rowRanges(ra) currently strips of extra columns, that are provided
when e.g.
creating ra from a GRangesList. Would it be possible to provide them
(as rowRanges for SummarizedExperiment does)?
MMorgan:
yes, something like as(ra, "GRangesList") should work
setAs("RaggedExperiment", "GRangesList", function(from) from@assays)
(I think GRangesList(ra) might then work 'out of the box').
Arguments given at the top of ?qreduceAssay:
qreduceAssay(x, query, simplify, i = 1, withDimnames = TRUE,
background = NA)
but further below it says:
qreduceAssay: A function accepting arguments score, range, and qrange:
I'm pulling a RaggedExperiment from TCGA:
> gbm <- curatedTCGAData::curatedTCGAData("GBM", "CNVSNP", FALSE)
> gbm.ra <- gbm[[1]]
> gbm.ra
class: RaggedExperiment
dim: 17818 154
assays(2): Num_Probes Segment_Mean
rownames: NULL
colnames(154): TCGA-02-0047-01A-01D-0182-01
TCGA-02-0055-01A-01D-0182-01 ...
TCGA-76-4931-01A-01D-1479-01 TCGA-28-2510-01A-01D-1694-01
colData names(0):
Looks like there is a small bug in the colnames<- replacement method:
> colnames(gbm.ra) <- TCGAutils::TCGAbarcode(colnames(gbm.ra), sample=TRUE)
Error in .local(x, ..., value = value) :
154 rows in value to replace 1104rows
> sessionInfo()
R Under development (unstable) (2019-01-07 r75958)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 9 (stretch)
Matrix products: default
BLAS/LAPACK: /usr/lib/libopenblasp-r0.2.19.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=C
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] parallel stats4 stats graphics grDevices utils
[7] datasets methods base
other attached packages:
[1] TCGAutils_1.3.5
[2] org.Hs.eg.db_3.7.0
[3] TxDb.Hsapiens.UCSC.hg19.knownGene_3.2.2
[4] GenomicFeatures_1.35.4
[5] AnnotationDbi_1.45.0
[6] curatedTCGAData_1.5.6
[7] CNVRanger_0.99.6
[8] RaggedExperiment_1.7.1
[9] MultiAssayExperiment_1.9.2
[10] SummarizedExperiment_1.13.0
[11] DelayedArray_0.9.6
[12] BiocParallel_1.17.6
[13] matrixStats_0.54.0
[14] Biobase_2.43.1
[15] GenomicRanges_1.35.1
[16] GenomeInfoDb_1.19.1
[17] IRanges_2.17.4
[18] S4Vectors_0.21.10
[19] BiocGenerics_0.29.1
loaded via a namespace (and not attached):
[1] httr_1.4.0 jsonlite_1.6
[3] bit64_0.9-7 AnnotationHub_2.15.4
[5] shiny_1.2.0 assertthat_0.2.0
[7] interactiveDisplayBase_1.21.0 BiocManager_1.30.4
[9] blob_1.1.1 GenomeInfoDbData_1.2.0
[11] Rsamtools_1.35.1 yaml_2.2.0
[13] progress_1.2.0 pillar_1.3.1
[15] RSQLite_2.1.1 lattice_0.20-38
[17] glue_1.3.0 digest_0.6.18
[19] promises_1.0.1 XVector_0.23.0
[21] rvest_0.3.2 htmltools_0.3.6
[23] httpuv_1.4.5.1 Matrix_1.2-15
[25] XML_3.98-1.16 pkgconfig_2.0.2
[27] biomaRt_2.39.2 zlibbioc_1.29.0
[29] purrr_0.2.5 xtable_1.8-3
[31] later_0.7.5 tibble_2.0.1
[33] magrittr_1.5 crayon_1.3.4
[35] mime_0.6 memoise_1.1.0
[37] xml2_1.2.0 tools_3.6.0
[39] prettyunits_1.0.2 hms_0.4.2
[41] stringr_1.3.1 bindrcpp_0.2.2
[43] Biostrings_2.51.2 compiler_3.6.0
[45] rlang_0.3.1 grid_3.6.0
[47] GenomicDataCommons_1.7.3 RCurl_1.95-4.11
[49] rstudioapi_0.9.0 rappdirs_0.3.1
[51] bitops_1.0-6 ExperimentHub_1.9.1
[53] DBI_1.0.0 curl_3.3
[55] R6_2.3.0 GenomicAlignments_1.19.1
[57] dplyr_0.7.8 rtracklayer_1.43.1
[59] bit_1.1-14 bindr_0.1.1
[61] readr_1.3.1 stringi_1.2.4
[63] Rcpp_1.0.0 tidyselect_0.2.5
library(GenomicRanges)
library(RaggedExperiment)
grl <- GRangesList(
sample1 = GRanges( c("chr1:1-10", "chr2:15-18", "chr2:25-34") ),
sample2 = GRanges( c("chr1:1-10", "chr2:11-18" , "chr2:25-36") ),
sample3 = GRanges( c("chr1:2-11", "chr2:14-18", "chr2:26-36") ),
sample4 = GRanges( c("chr1:1-12", "chr2:18-35" ) ),
sample5 = GRanges( c("chr1:1-12", "chr2:11-17" , "chr2:26-34") ) ,
sample6 = GRanges( c("chr1:1-12", "chr2:12-18" , "chr2:25-35") )
)
ra <- RaggedExperiment(grl)
Some arbitrary colData annotation:
> colData(ra)$colanno <- seq_len(ncol(ra))
> colData(ra)
DataFrame with 6 rows and 1 column
colanno
<integer>
sample1 1
sample2 2
sample3 3
sample4 4
sample5 5
sample6 6
Ok.
Some arbitrary rowData annotation:
> rowData(ra)$rowanno <- LETTERS[seq_len(nrow(ra))]
> rowData(ra)
Error in (function (classes, fdef, mtable) :
unable to find an inherited method for function ‘getListElement’ for signature ‘"GRanges"’
Also, colData now returns the row annotation (??):
> colData(ra)
DataFrame with 6 rows and 1 column
rowanno
<character>
1 A
2 B
3 C
4 D
5 E
6 F
> sessionInfo()
R version 3.5.0 (2018-04-23)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.3
Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] parallel stats4 stats graphics grDevices utils datasets
[8] methods base
other attached packages:
[1] RaggedExperiment_1.5.0 GenomicRanges_1.33.1 GenomeInfoDb_1.17.0
[4] IRanges_2.15.4 S4Vectors_0.19.2 BiocGenerics_0.27.0
loaded via a namespace (and not attached):
[1] lattice_0.20-35 matrixStats_0.53.1
[3] bitops_1.0-6 grid_3.5.0
[5] zlibbioc_1.27.0 XVector_0.21.0
[7] Matrix_1.2-14 BiocParallel_1.15.0
[9] tools_3.5.0 Biobase_2.41.0
[11] RCurl_1.95-4.10 DelayedArray_0.7.0
[13] compiler_3.5.0 SummarizedExperiment_1.11.1
[15] GenomeInfoDbData_1.1.0
>
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.