[BUG]: Error executing process > 'NF_FLU:ILLUMINA:SUBTYPING_REPORT (1)' about nf-flu HOT 3 OPEN

Hi @PierreLyons Thanks for taking the time to report your issue!

Would you happen to be able to attach/copy-paste the contents of parse_influenza_blast_results.log for this analysis?

The parse_influenza_blast_results.py Python script makes use of Polars, Pandas and NumPy which may be trying to use functions that rely on certain CPU instructions. Or there could just be a bug in the script and the full stack trace might be helpful.

Have you tried Conda/Mamba instead of Podman/Docker?

from nf-flu.

Hi @peterk87

Thanks for the quick response.

I haven't tried Conda yet, it's next on my list to try. I'll update once I do.

I'm also struggling to find the parse_influenza_black_results.log file, any ideas where it is stored? Thanks.

from nf-flu.

Update regarding this issue:

Note: the same issue happens using either Docker or Conda/Mamba.

I've managed to isolate the issue to the use of polars, which seems to have been added to parse_influenza_blast_results.py in revision 3.2.0, which is when the pipeline breaks on my system. polars uses AVX, which my cpu doesn't support. The polars team has created a legacy version of polars (polars-lts-cpu) which was complied without AVX, and can be installed via pip (not available within conda-forge).

As a quick patch, I've manually created the conda env used by the subtyping_report.nf module with all dependencies except for polars, and then pip installed polars-lts-cpu within that environment. I then simply point the subtyping_report.nf module to the custom environment.

This is working well (almost, see below) now as a patch. I've also requested newer hardware, which I think is the most sensible solution to this issue.

The SUBTYPING_REPORT module completed successfully, but then the SOFTWARE_VERSIONS module threw an error.

I will describe it quickly here as well as my fix, but let me know if you'd like me to open a new issue. I'm not sure if my above fix caused this new issue.

dumpsoftwareversions.py caused this error (truncated for readability):
ERROR ~ Error executing process > 'NF_FLU:ILLUMINA:SOFTWARE_VERSIONS (1)'

Caused by:
Process NF_FLU:ILLUMINA:SOFTWARE_VERSIONS (1) terminated with an error exit status (1)

Command executed [/home/vitalite/.nextflow/assets/CFIA-NCFAD/nf-flu/./workflows/../modules/nf-core/modules/custom/dumpsoftwareversions/templates/dumpsoftwareversions.py]:

[ ... error truncated for readability ... ]

yaml.scanner.ScannerError: mapping values are not allowed here
in "collated_versions.yml", line 2, column 13

For reference, here are the first two lines of collated_versions.yml (second line truncated for readability):
"NF_FLU:ILLUMINA:CAT_ILLUMINA_FASTQ":
cat: Usage: cat [OPTION]... [FILE]... Concatenate FILE(s) to standard output. W [...]

This issue seems to be with the ":" after Usage.

My fix was to add a sed to remove colons from the echo commands at lines 103 and 104 in the nf-flu/modules/local/cat_illumina_fastq.nf file:

original:
cat: $(echo $(cat --help 2>&1) | sed 's/ (.//')
gzip: $(echo $(gzip --help 2>&1) | sed 's/ (.//')

modified:
cat: $(echo $(cat --help 2>&1) | sed 's/ (.//' | sed 's/://')
gzip: $(echo $(gzip --help 2>&1) | sed 's/ (.//' | sed 's/://')

This has solved the issue and now the pipeline runs (note: I can only run the pipeline with Conda).

Once again. let me know if I should open a new issue.
Thanks,
Pierre

from nf-flu.