• change input_check.nf:create_fastq_channel():meta.id back to sample name
• supported formats:
  • ome.tif
  • moved zarr support to separate ticket
• change sample sheet format
  • schema change
  • document all the places that a sample sheet format change has to be made because I forget
    • Just in assets/schema_input.json
  • So should ashlar use all ome.tif or zarr images in the input directory as input?
    • doing this as default behavior
    • need to only use allowed file types
• make input parameter for each style of sample sheet
  • one row per sample
    • update mcmicro.nf
    • update nextflow_schema.json
    • rename assets/schema_input.json
    • nextflow.config
    • lib/WorkflowMain.groovy
  • one row per sample per cycle
    • update mcmicro.nf to handle either input sample sheet
    • update nextflow_schema.json
    • create assets/schema_input_cycle.json
    • update lib/WorkflowMain.groovy
    • update nextflow.config
  • use the '--input' parameter and for it check for cycle column to set input type
    • my concern with this is that I don't know how we'd get automatic validation because we have to set a schema for each input parameter
    • UPDATE: I thought I could get this working by setting the input parameter --input_sample or --input_cycle to the path given for --input and then use the correct one of those as input to Channel.fromSamplesheet(), but it doesn't work. It looks like fromSamplesheet() only accepts an input parameter name and it only gets the value of that input parameter from session.params. I tried updating the value of session.params['input_cycle'] from within the workflow, but this didn't work. It still had the default value defined in nextflow.config.
    • we're not going to support this at least at the moment
• question: what are better input parameter names; currently:
  • input_sample
  • input_cycle
• question: do the workflow inputs need to be added to nextflow.config?
  • I don't see any changes for the current parameters
  • Yes, I had to add them to pass lint tests
• marker_sheet as input parameter
  • add to nextflow_schema.json
  • create schma file assets/schema_marker.json
  • create channel from parameter in mcmicro.nf
  • question: is there a marker sheet for the tonsil data?
    • there is, Jeremy will send it
  • question: are the only columns for a marker sheet: channel_number, cycle_number, marker_name, Filter, excitation_wavelength, emission_wavelength?
    • needs to be discussed with larger group
  • question: which columns are required?
    • cycle, channel and marker_name
• question: my code assumes input sheet order is fixed. Is this a valid assumption or should I be expecting the columns in any order?
  • I ran into this when making ashlar input and validating the marker sheet. It was easy to make it work for any column order, so I just added it.
• nf-validation:
  • add paramsHelp
    • already there, but was in main.nf, so I didn't notice
• we would like more complex validation that checks associated files/directories. Can we do that with nf-validation?
  • it doesn't look like it
  • create new local subworkflow modules for the following validation:
    • change to subworkflow
      • made subworkflow for marker sheet validation
      • add sample sheet validation to subworkflow
      • clean-up new validation
      • clean-up (remove) old validation
    • validation:
      • uniqueness of marker name in marker sheet
      • uniqueness of (cycle, channel) tuple in marker sheet
      • cycle and channel are 1-based so 0 should throw an exception
      • cycle and channel cannot have skips and must be in order
      • validate that every sample has all the same cycle number values, and that those values match with the marker sheet 1:1
    • python exceptions should trigger Nextflow error messages
      • low priority because we'll probably be moving these python functions to groovy (#2)
• question: any additional input validation on:
  • input_sample?
  • input_cycle?
  • marker_sheet?
• ext.args for ashlar input parameters
  • handle ffp & dfp
  • moved to workflow parameters
• question: what should the workflow output?
  • currently just passing the last module output file as placeholder
  • be sure to add an output.md describing it (required by lint tests)
• tests
  • nf-test
    • basic end-to-end test
    • test md5sum of each module's output
  • pytest end-to-end workflow test with md5sum check on module output files
  • some lint tests are failing
    • files_exist
    • files_unchanged
    • merge_markers
    • multiqc_config
    • nextflow_config
      • asked on nf-core:mcmicro Slack channel. Waiting for response
    • readme
      • posted question on Slack linting channel about this
    • schema_description
• fix lint warnings
• bug fix
  • not handling 1 row per sample per cycle inputs correctly. All rows with the same sample name need to go to one run of ashlar.
  • same issue for 1 row per sample inputs
• add image correction support to 1 row per sample case
  • manual
  • basicpy

ashlar should work after next update. I'll try again then.

Description of feature

adding

"format": "file-path",
"exists": true,

works for files, but changing to "format": directory-path doesn't work for image_directory when the path is a URL.