sanjaynagi / ampseeker Goto Github PK

• bbduk is not a dependency, lets avoid

with test dataset of amplicon data

AmpSeeker?

Thoughts welcome.

or results/ directory.

Symlink should open straight to web-browser when linked to.

For example:

• Things like filtering on depth

• Heterozygosity (Hardy-weinberg)

• Need inspiration for other thresholds, worth looking back at Ag1000G QC steps, or any other AmpSeq pipelines/analyses.

We will need to implement a species ID script for AgamDao.

This will be the first protocol-specific bit of analysis, and so we will need to think about the best way to approach this. There could simply be a series of options in the config for each protocol AmpSeeker supports.

Modules:
  AgamDao: True #anopheles
  GRC1: False #plasmodium

TODO

• Species ID
• KDR haplotypes/diplotypes
• ???

add coverage by sample in the notebook

Need a rule at the start of the workflow to convert and demultiplex BCL files from the Illumina miseq output directory to fastq.
The command should be something like -

bcl-convert --bcl-input-directory {illumina_out_dir} --output-directory resources/reads --sample-sheet {illumina_out_dir}/SampleSheet.csv

• we only test starting from reads

This will involve getting bcl-convert to produce fastqs for the index read. however, I could only get bcl2fastq to do it, not bcl-convert, so we might want to change to bcl2fastq, the older software.

We can then use fastqc or fastp on the index reads.

A script which after demultiplexing, counts reads per sample and makes a heatmap of reads per well of each input plate.

Use a jupyter notebook with papermill to explore read data in IGV

• using ipyleaflet as in https://github.com/anopheles-genomic-surveillance/selection-atlas

Workflow throws a
ChildIOException: File/directory is a child to another output: when provided with an Illumina data folder.

• papermill / nb style
• Coverage at each amplicon
• Coverage at each SNP

• Calculate allele frequencies
• Plot as heatmap with plotly

• Replace mpileup? eventually.
• For now keep in gatk.smk

Sometimes samples will have zero data, which makes the pipeline fail.

To handle this, the pipeline should start with a checkpoint which evaluates what samples actually have data and uses these to run the rest of the pipeline with.

on Docs book home-page

• Its a really useful tool https://multiqc.info/

• example usage in a snakemake workflow here - https://github.com/sanjaynagi/rna-seq-pop/blob/master/workflow/rules/qc.smk

What columns are required?

• Need to use diff conda env (try selection-atlas env)

Pin to python=3.8/3.9

• Need to download larger subset of data (~100, 200 samples at least?)
• And find corresponding Ag1000g sampleIDs so we can match them up
• evaluate genotype concordance

@ChabbyTMD @eddUG

I was having a think, and I think it should be possible to use Jupyter-book within snakemake to build a private web-site, which contains all of the results of the workflow for each users analysis.

This would be really cool, imagine that you wouldn't even need to look in the results folder, one could simply open the webpage and explore all the results that way. If we make the analyses in papermill/Jupyter-notebooks, this shouldn't be too complicated!

(Ill explain in our next meeting, but jupyter-book can basically take as input a load of jupyter notebooks and build a website from it).

add plasmodium/bacteria/virus and mosquito

Currently, the pipeline has a rule which splits the bcftools merge step into two groups, as when running more than 1000 samples, bcftools merge fails. It also means the pipeline can fail if there are less than 1000 samples.

We should write this so that this is automated, I.e merging is only done in 2 steps if there are more than 1000 samples.

It is not documentation...

Need to change to bcl2fastqe

As well as the results Jupyter-book, we also want a jupyter book which will be public, and contain the documentation, for things like

• setting up AmpSeeker (config, input files)
• contributing
• troubleshooting

This will be hosted within the AmpSeeker repo using github pages.

With snpeff or something else

to repo

Im getting a conda error when trying to run the workflow due to the qualimap wrapper.

Add badge for github actions CI build (https://shields.io/category/build)

and remove all the snakemake talk from the README, maybe moving it to the wiki page.

Currently, the pipeline will fail if samples have 001 in their sampleID

As different parts of the analysis are optional, the toc will also need to be generated dynamically, within the workflow.

• need to ensure we are implementing some sensible QC steps, I dont quite know what these would look like atm.

:)

(where possible)

controlled in the config.yaml

Basically the VCF genotypes but as a pandas dataframe,

Samples as columns, SNPs as rows.

• On called variants
• Should be a jupyter notebook run by papermill, plotting done with plotly to be interactive

We should keep the snakefile tidy, and so have an input function for rule all, which resides in a rule file called common.smk. This is good practice in snakemake.

This function will determine which output files we want to produce, based on the config.yaml

e.g px.box(df, x='x', y='y', template='simplewhite')

Looks are lot better than the grey