Test how variable numbers of inputs and outputs work about modules HOT 5 CLOSED

Yeah, this is going to be tricky. Pipelines will want to use the module command in different ways so we will have to try and anticipate different numbers/types of inputs and outputs. Should this be done on a case-by-case basis? e.g. add the option for a another input/output whilst maintaining backward compatability?

With DSL1 the way I have provided conditional inputs is:
https://github.com/nf-core/chipseq/blob/21be3149542cdc84431e12d1e092359058aed32a/main.nf#L424

and conditional outputs as:
https://github.com/nf-core/chipseq/blob/21be3149542cdc84431e12d1e092359058aed32a/main.nf#L1087

Note all of the files dont have to be created as an output here and so NF wont fail if they arent. Maybe the solution is to have all outputs from a module in a single channel and then filter as required but this could get messy...

from modules.

I think DSLv2 can have named outputs, right? So I'm hoping that if pipelines specify the output channels that they are using, they can ignore the ones that they don't need. Then we can just allow those channels to be empty and I think everything should be fine... 🤞

The conditional input ifEmpty() thing shouldn't be necessary... I think... because with modules you don't specify the input channel name. Not sure though.

I think the one channel for everything isn't necessary if we use optional outputs and named outputs in the workflows.

Make sense? Bit scatterbrained today sorry - say if the above is jumbled jibberish 😆

from modules.

No. That makes sense. Like most things "modules" this will need a play around to work out how best to proceed. I need to fix up and look sharp with DSL2 too. Maybe we can try with a simple TrimGalore implementation?

from modules.

Yes I agree - we should really get some "basic" tests for this and play around a bit to get a feeling for how this behaves. During the discussion we all kind of agreed its got lots of potential for problems arising, which is why Phil created the issue here 👍

from modules.

We have a requirement to emit every output file which solves this issue. Where required, we are using the optional declaration for optional outputs.

We still don't have a proper solution for optional inputs but this will have to be announced officially on the Nextflow end

from modules.