sydneybiox / scjoint Goto Github PK

Python 2.63% R 0.27% Jupyter Notebook 97.10%

scjoint's Issues

an error in `model_stage1.train`

Hi, authors,
I got an error when I ran python3 main.py,just like the follow. I am confused of this problem,I would appreciate it if you could reply to me soon, Thanks!!

ValueError: not enough values to unpack (expected 2, got 0)

Hi ,

I hope you're doing well. I wanted to inform you that I encountered an error while running the file main.py. The error occurs in the progress_bar function of the utils.py file. Specifically, the function fails to retrieve the terminal size, resulting in a "ValueError: not enough values to unpack" error.

Thank you very much for your work on this project and your attention to this matter.

Best regards,

settings for scJoint

Hi,

I am wondering, can somebody help me with choosing the settings in config.py ?
I have integrated scRNA-Seq data with several differing factors (different technologies, stim and ctrl, different Age). I also have scATAC data, but from a completely different batch of samples.
After running scJoint according to the 10X tutorial notebook, the two datasets do not look nicely overlayed in the tSNE-plot.
Is there maybe a possibility to use pre-calculated embedding-values for RNA, so that the clustering/plotting is the same as in Seurat and the ATAC cells will be put in this embedding?
Or can I change the resolution of the tSNE somehow?

As I am a newby to python and this kind of analysis, I'd appreciate any help.

Thanks!

object of type 'S4' is not subsettable in write_h5_scJoint

Hi,

I used the test rds data downloaded from the data_10x folder and tried to combine scATAC and scRNA data using the following scripts:

sce_10xPBMC_atac <- readRDS("/scratch/hy17471/software/scJoint/data_10x/sce_10xPBMC_atac.rds")
sce_10xPBMC_rna <- readRDS("/scratch/hy17471/software/scJoint/data_10x/sce_10xPBMC_rna.rds")

common_genes <- intersect(rownames(sce_10xPBMC_atac),
rownames(sce_10xPBMC_rna))
length(common_genes)

exprs_atac <- logcounts(sce_10xPBMC_atac[common_genes, ])
exprs_rna <- logcounts(sce_10xPBMC_rna[common_genes, ])

write_h5_scJoint(exprs_list = list(rna = exprs_rna,
atac = exprs_atac),
h5file_list = c(paste0(output_dir,'/exprs_10xPBMC_test_rna.h5'),
paste0(output_dir,'/exprs_10xPBMC_test_atac.h5')))

but it returns:
Error in .Primitive("[")(new("HDF5RealizationSink", dim = c(9841L, 15463L :
object of type 'S4' is not subsettable

Would you help to figure it out. Thanks

Best,
Haidong

an error in model_stage1.train(epoch) by the test 10X_data

Hello,authors:
I tested the test data_10X which from author and got the same error with my data. The error is the follow:

I am so confused that why the test data also came out the same problem, I would appreciate it if you could reply to me soon. Thanks!

How are cells paired?

Hi SydneyBioX researchers,

I am not sure if I understand your code very well. I am confused on one thing. How cells from RNA-seq and ATAC-seq are paired. Is it achieved through NNDR? This is where I get lost. Thank you so much.

ValueError at the "write embeddings [stage 3]" step for RNA

Hello,

I am running scJoint on unpaired scRNA-seq and scATAC-seq data for label transfer from RNA to ATAC modalities. I get all the output correctly except for the RNA_embeddings because of the error below when running main.py :

"Traceback (most recent call last):
File "main.py", line 56, in
main()
File "main.py", line 47, in main
model_stage3.write_embeddings()
File "/Analysis/ATAC/scJoint/util/trainingprocess_stage3.py", line 210, in write_embeddings
rna_embedding = rna_embedding / norm(rna_embedding, axis=1, keepdims=True)
File "/anaconda3/lib/python3.8/site-packages/scipy/linalg/misc.py", line 140, in norm
a = np.asarray_chkfinite(a)
File "/anaconda3/lib/python3.8/site-packages/numpy/lib/function_base.py", line 488, in asarray_chkfinite
raise ValueError(
ValueError: array must not contain infs or NaNs"

Can you please help with that error ?

Thank you,

Kind regards,
Sébastien

Questions related to scanpy data

Hi, I wonder is it possible for us to utilize this model based on scanpy data? Thanks.

Is it possible to use the tool on 4 samples?

Hello SydneyBioX lab!

I have a question about your tool! I have 4 data: scRNA WT, scATAC WT, scRNA MUT and scATAC MUT.
Would it be possible to use your tool on these 4 samples altogether? Or maybe is it better to integrate all the scRNA WT and MUT, integrate all the scATAC WT and MUT and then run your tool?

Thanks

An Indentation error in knn.py

In the #test part of util/knn.py, the calculation of knn accuracy should be indented, maybe.

CUDA error in stage 3

Hi scJoint team,

In our projects the scJoint worked smoothly in the stage 1 and 2, but threw error on the stage3.
It appeard to me that the all of the loss value are getting incredibly large.

Many thanks for your insights!

Following are the output for stage 3:


Training start [Stage3]
num_workers: 0
load npz matrix: /home/shao/Desktop/Projects/X/10x_ATACseq/scJoint/data_hp_wt/wt_hp_rna.npz
load npz matrix: /home/shao/Desktop/Projects/X/10x_ATACseq/scJoint/data_hp_wt/wt_hp_atac.npz
Epoch: 0
LR is set to 0.01
LR is set to 0.01
 [============ 29/29 =========>.]    Step: 157ms | Tot: 4s393ms | encoding_loss: 12.817, rna_loss: 59.222, center_loss: 5.881
Epoch: 1
 [============ 29/29 =========>.]    Step: 156ms | Tot: 4s366ms | encoding_loss: 919.316, rna_loss: 27529.147, center_loss: 120.546
Epoch: 2
 [================ 29/29 =====>.]    Step: 158ms | Tot: 4s339ms | encoding_loss: 11958.841, rna_loss: 204954.188, center_loss: 413.1Epoch: 3
 [=================== 29/29 ==>.]    Step: 156ms | Tot: 4s349ms | encoding_loss: 260700.759, rna_loss: 7364164.912, center_loss: 254Epoch: 4
 [========================== 29/29   Step: 158ms | Tot: 4s335ms | encoding_loss: 137907357.234, rna_loss: 5798996849.379, center_losEpoch: 5.433
 [============================>.] 29/29 p: 156ms | Tot: 4s368ms | encoding_loss: 25037452131.310, rna_loss: 874840569008.552, centerEpoch: 658529.145
 [============================>.]    Step:  29/29  Tot: 4s324ms | encoding_loss: 104636323585765.516, rna_loss: 3276948089484535.000Epoch: 7_loss: 34642031.474
 [============================>.]    Step: 157 29/29 t: 4s347ms | encoding_loss: 2187132071060303.500, rna_loss: 73972744484661184.0Epoch: 8er_loss: 203889508.828
 [============================>.]    Step: 158ms | T 29/29 02ms | encoding_loss: 972514208877650816.000, rna_loss: 46906742034446524Epoch: 9 center_loss: 4715636888.276
 [============================>.]    Step: 157ms | Tot: 4s3 29/29 encoding_loss: 361596709780774846464.000, rna_loss: 11707894052183Epoch: 10.000, center_loss: 68285345968.552
 [============================>.]    Step: 157ms | Tot: 4s388ms  29/29 ing_loss: 32664000407402539122688.000, rna_loss: 143521479561Epoch: 112480.000, center_loss: 123240449412.414
 [============================>.]    Step: 159ms | Tot: 4s444ms | en 29/29 loss: 8485904532940365594886144.000, rna_loss: 3706595062Epoch: 1209562368.000, center_loss: 125000001182.897
 [============================>.]    Step: 160ms | Tot: 4s436ms | encodin 29/29  1186645034441254863890284544.000, rna_loss: 4430101Epoch: 131753024004096.000, center_loss: 125000002877.793
 [============================>.]    Step: 160ms | Tot: 4s479ms | encoding_los 29/29 50627509908939357502308352.000, rna_loss: 43719Epoch: 14617924641928249344.000, center_loss: 125000002595.310
 [============================>.]    Step: 159ms | Tot: 4s428ms | encoding_loss: 745 29/29 26601367427467389698048.000, rna_loss: 23Epoch: 15674412783426934507831296.000, center_loss: 125000002312.828
 [============ 29/29 =========>.]    Step: 159ms | Tot: 4s458ms | encoding_loss: inf, rna_loss: inf, center_loss: nan               Epoch: 16er_loss: 125000001536.000 000, center_loss: 125000000853.333
Traceback (most recent call last):   Step: 161ms | Tot: 2s89ms | encoding_loss: inf, rna_loss: nan, center_loss: nan
  File "main.py", line 56, in <module>
    main()
  File "main.py", line 44, in main
    model_stage3.train(epoch)
  File "/home/shao/Desktop/Projects/X/10x_ATACseq/scJoint/util/trainingprocess_stage3.py", line 152, in train
    encoding_loss.backward(retain_graph=True)
  File "/home/shao/miniconda3/envs/env_scjoint/lib/python3.8/site-packages/torch/tensor.py", line 195, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/shao/miniconda3/envs/env_scjoint/lib/python3.8/site-packages/torch/autograd/__init__.py", line 97, in backward
    Variable._execution_engine.run_backward(
RuntimeError: CUDA error: device-side assert triggered

Stage 1 and knn worked fine

Start time:  09:20:49
Training start [Stage1]
num_workers: 0
load npz matrix: /home/shao/Desktop/Projects/X/10x_ATACseq/scJoint/data_hp_wt/wt_hp_rna.npz
load npz matrix: /home/shao/Desktop/Projects/X/10x_ATACseq/scJoint/data_hp_wt/wt_hp_atac.npz
Epoch: 0
LR is set to 0.01
LR is set to 0.01
 [============ 29/29 =========>.]    Step: 152ms | Tot: 4s236ms | encoding_loss: 5.734, rna_loss: 1.521
Epoch: 1
 [============ 29/29 =========>.]    Step: 154ms | Tot: 4s270ms | encoding_loss: 3.825, rna_loss: 0.552
Epoch: 2
 [============ 29/29 =========>.]    Step: 152ms | Tot: 4s250ms | encoding_loss: 3.643, rna_loss: 0.281
Epoch: 3
 [============ 29/29 =========>.]    Step: 155ms | Tot: 4s256ms | encoding_loss: 3.586, rna_loss: 0.172
Epoch: 4
 [============ 29/29 =========>.]    Step: 157ms | Tot: 4s297ms | encoding_loss: 3.541, rna_loss: 0.128
Epoch: 5
 [============ 29/29 =========>.]    Step: 154ms | Tot: 4s283ms | encoding_loss: 3.507, rna_loss: 0.091
Epoch: 6
 [============ 29/29 =========>.]    Step: 155ms | Tot: 4s290ms | encoding_loss: 3.483, rna_loss: 0.075
Epoch: 7
 [============ 29/29 =========>.]    Step: 153ms | Tot: 4s256ms | encoding_loss: 3.462, rna_loss: 0.057
Epoch: 8
 [============ 29/29 =========>.]    Step: 155ms | Tot: 4s303ms | encoding_loss: 3.454, rna_loss: 0.044
Epoch: 9
 [============ 29/29 =========>.]    Step: 156ms | Tot: 4s327ms | encoding_loss: 3.435, rna_loss: 0.038
Epoch: 10
 [============ 29/29 =========>.]    Step: 156ms | Tot: 4s351ms | encoding_loss: 3.426, rna_loss: 0.032
Epoch: 11
 [============ 29/29 =========>.]    Step: 162ms | Tot: 4s337ms | encoding_loss: 3.410, rna_loss: 0.027
Epoch: 12
 [============ 29/29 =========>.]    Step: 160ms | Tot: 4s334ms | encoding_loss: 3.399, rna_loss: 0.026
Epoch: 13
 [============ 29/29 =========>.]    Step: 159ms | Tot: 4s356ms | encoding_loss: 3.382, rna_loss: 0.022
Epoch: 14
 [============ 29/29 =========>.]    Step: 155ms | Tot: 4s311ms | encoding_loss: 3.370, rna_loss: 0.019
Epoch: 15
 [============ 29/29 =========>.]    Step: 155ms | Tot: 4s313ms | encoding_loss: 3.363, rna_loss: 0.017
Epoch: 16
 [============ 29/29 =========>.]    Step: 153ms | Tot: 4s284ms | encoding_loss: 3.347, rna_loss: 0.016
Epoch: 17
 [============ 29/29 =========>.]    Step: 152ms | Tot: 4s295ms | encoding_loss: 3.338, rna_loss: 0.015
Epoch: 18
 [============ 29/29 =========>.]    Step: 153ms | Tot: 4s305ms | encoding_loss: 3.339, rna_loss: 0.014
Epoch: 19
 [============ 29/29 =========>.]    Step: 154ms | Tot: 4s319ms | encoding_loss: 3.331, rna_loss: 0.014
Write embeddings
 [============ 26/26 =========>.]    Step: 40ms | Tot: 2s639ms | write embeddings and predictions for db:wt_hp_rna
 [============ 30/30 ==========>]    Step: 48ms | Tot: 3s94ms | write embeddings and predictions for db:wt_hp_atac
Stage 1 finished:  09:22:28
KNN
[KNN] Read RNA data
[KNN] Read ATAC data
[KNN] Build Space
[KNN] knn
KNN finished:  09:22:30

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.