Comments (7)
jmfernandez
commented on August 18, 2024
1
Thanks for the feedback, I'm trying this one later.
In addition to the above comment, I still cannot run WfExS with an RO-Crate directly from disk. I downloaded this workflow which is same one as the file example I'm trying to run. I get a similar error to above, where is says no engine recognised.
2023-07-10 09:55:19,258 - [WARNING] Unable to process CWL entrypoint /root/wfexs-backend-test_WorkDir/72a0e402-9336-478c-8693-f1a71ffa6f5b/workflow [Errno 21] Is a directory: '/root/wfexs-backend-test_WorkDir/72a0e402-9336-478c-8693-f1a71ffa6f5b/workflow' Traceback (most recent call last): File "/root/WfExS-backend/WfExS-backend.py", line 21, in <module> main() File "/root/WfExS-backend/wfexs_backend/__main__.py", line 1122, in main stagedSetup = wfInstance.stageWorkDir() File "/root/WfExS-backend/wfexs_backend/workflow.py", line 1995, in stageWorkDir self.materializeWorkflowAndContainers(offline=offline, ignoreCache=ignoreCache) File "/root/WfExS-backend/wfexs_backend/workflow.py", line 1243, in materializeWorkflowAndContainers self.setupEngine(offline=offline, ignoreCache=ignoreCache) File "/root/WfExS-backend/wfexs_backend/workflow.py", line 1201, in setupEngine self.fetchWorkflow( File "/root/WfExS-backend/wfexs_backend/workflow.py", line 1176, in fetchWorkflow raise WFException( wfexs_backend.workflow.WFException: No engine recognized a workflow at file:///root/test_wfexs_dir
from wfexs-backend.
dcl10
commented on August 18, 2024
1
Hi @jmfernandez. I hope you had a nice weekend. As per your email, I've tried WfExS with the files above. I found 2 thing:
1. You need to add `groovy_parser`, `lark` and `pygments` to your `requirements.txt` file.Hi @dcl10, did you pull this morning all the changes and updated the requirements.txt ? Because this last was updated in the commit from 15 hours ago,
WfExS-backend/requirements.txt
Line 24 in fde8158
Hi @jmfernandez, I pulled this morning but I pulled the wrong tag haha. Thanks for pointing this out :)
from wfexs-backend.
dcl10
commented on August 18, 2024
Hi @jmfernandez. I hope you had a nice weekend. As per your email, I've tried WfExS with the files above. I found 2 thing:
- You need to add
groovy_parser,larkandpygmentsto yourrequirements.txtfile. - I got the following traceback when attempting to stage the workflow:
2023-07-10 09:42:38,814 - [ERROR] Failed to parse initial file sec-hutchx86.cwl with groovy parser
Traceback (most recent call last):
File "/root/WfExS-backend/wfexs_backend/nextflow_engine.py", line 383, in identifyWorkflow
) = analyze_nf_content(firstPathContent, only_names=only_names)
File "/root/WfExS-backend/wfexs_backend/utils/groovy_parsing.py", line 643, in analyze_nf_content
t_tree = parse_and_digest_groovy_content(content)
File "/root/WfExS-backend/wfexs_backend/utils/groovy_parsing.py", line 633, in parse_and_digest_groovy_content
tree = parse_groovy_content(content)
File "/root/WfExS-backend/.pyWEenv/lib/python3.10/site-packages/groovy_parser/parser.py", line 158, in parse_groovy_content
raise pe
File "/root/WfExS-backend/.pyWEenv/lib/python3.10/site-packages/groovy_parser/parser.py", line 153, in parse_groovy_content
tree = parser.parse(
File "/root/WfExS-backend/.pyWEenv/lib/python3.10/site-packages/lark/lark.py", line 645, in parse
return self.parser.parse(text, start=start, on_error=on_error)
File "/root/WfExS-backend/.pyWEenv/lib/python3.10/site-packages/lark/parser_frontends.py", line 96, in parse
return self.parser.parse(stream, chosen_start, **kw)
File "/root/WfExS-backend/.pyWEenv/lib/python3.10/site-packages/lark/parsers/earley.py", line 266, in parse
to_scan = self._parse(lexer, columns, to_scan, start_symbol)
File "/root/WfExS-backend/.pyWEenv/lib/python3.10/site-packages/lark/parsers/earley.py", line 237, in _parse
to_scan = scan(i, token, to_scan)
File "/root/WfExS-backend/.pyWEenv/lib/python3.10/site-packages/lark/parsers/earley.py", line 214, in scan
raise UnexpectedToken(token, expect, considered_rules=set(to_scan), state=frozenset(i.s for i in to_scan))
lark.exceptions.UnexpectedToken: Unexpected token Token('FLOATING_POINT_LITERAL', (Token.Literal.Number.Float, '0', '0')) at line 1, column 15.
Expected one of:
* WHILE
* THREADSAFE
* NL
* SUPER
* BOOLEAN
* FLOAT
* FOR
* CLASS
* INT
* TRAIT
* SWITCH
* ELSE
* GOTO
* IMPORT
* RETURN
* TRANSIENT
* ABSTRACT
* PRIVATE
* NATIVE
* AT
* INTERFACE
* EXTENDS
* IMPLEMENTS
* DO
* DEFAULT
* THIS
* CAPITALIZED_IDENTIFIER
* FINALLY
* THROWS
* IN
* AS
* LPAREN
* FINAL
* CATCH
* SYNCHRONIZED
* LONG
* VOLATILE
* BOOLEAN_LITERAL
* IF
* CHAR
* CONST
* BREAK
* CONTINUE
* PUBLIC
* INSTANCEOF
* ENUM
* VOID
* STATIC
* NULL_LITERAL
* PROTECTED
* CASE
* BYTE
* DOUBLE
* DEF
* STRICTFP
* VAR
* PACKAGE
* STRING_LITERAL
* ASSERT
* LT
* THROW
* SHORT
* GSTRING_BEGIN
* TRY
* IDENTIFIER
* NEW
Traceback (most recent call last):
File "/root/WfExS-backend/WfExS-backend.py", line 21, in <module>
main()
File "/root/WfExS-backend/wfexs_backend/__main__.py", line 1122, in main
stagedSetup = wfInstance.stageWorkDir()
File "/root/WfExS-backend/wfexs_backend/workflow.py", line 1995, in stageWorkDir
self.materializeWorkflowAndContainers(offline=offline, ignoreCache=ignoreCache)
File "/root/WfExS-backend/wfexs_backend/workflow.py", line 1243, in materializeWorkflowAndContainers
self.setupEngine(offline=offline, ignoreCache=ignoreCache)
File "/root/WfExS-backend/wfexs_backend/workflow.py", line 1201, in setupEngine
self.fetchWorkflow(
File "/root/WfExS-backend/wfexs_backend/workflow.py", line 1176, in fetchWorkflow
raise WFException(
wfexs_backend.workflow.WFException: No engine recognized a workflow at file:///root/hutch/workflows/sec-hutchx86.cwl
from wfexs-backend.
dcl10
commented on August 18, 2024
In addition to the above comment, I still cannot run WfExS with an RO-Crate directly from disk. I downloaded this workflow which is same one as the file example I'm trying to run. I get a similar error to above, where is says no engine recognised.
2023-07-10 09:55:19,258 - [WARNING] Unable to process CWL entrypoint /root/wfexs-backend-test_WorkDir/72a0e402-9336-478c-8693-f1a71ffa6f5b/workflow [Errno 21] Is a directory: '/root/wfexs-backend-test_WorkDir/72a0e402-9336-478c-8693-f1a71ffa6f5b/workflow'
Traceback (most recent call last):
File "/root/WfExS-backend/WfExS-backend.py", line 21, in <module>
main()
File "/root/WfExS-backend/wfexs_backend/__main__.py", line 1122, in main
stagedSetup = wfInstance.stageWorkDir()
File "/root/WfExS-backend/wfexs_backend/workflow.py", line 1995, in stageWorkDir
self.materializeWorkflowAndContainers(offline=offline, ignoreCache=ignoreCache)
File "/root/WfExS-backend/wfexs_backend/workflow.py", line 1243, in materializeWorkflowAndContainers
self.setupEngine(offline=offline, ignoreCache=ignoreCache)
File "/root/WfExS-backend/wfexs_backend/workflow.py", line 1201, in setupEngine
self.fetchWorkflow(
File "/root/WfExS-backend/wfexs_backend/workflow.py", line 1176, in fetchWorkflow
raise WFException(
wfexs_backend.workflow.WFException: No engine recognized a workflow at file:///root/test_wfexs_dir
from wfexs-backend.
jmfernandez
commented on August 18, 2024
Hi @jmfernandez. I hope you had a nice weekend. As per your email, I've tried WfExS with the files above. I found 2 thing:
1. You need to add `groovy_parser`, `lark` and `pygments` to your `requirements.txt` file.
Hi @dcl10, did you pull this morning all the changes and updated the requirements.txt ? Because this last was updated in the commit from 15 hours ago,
WfExS-backend/requirements.txt
Line 24 in fde8158
from wfexs-backend.
jmfernandez
commented on August 18, 2024
Hi again!
I have identified the source of the groovy-parser issue you found when it could be reproduced by @paulaidt in a fresh installation. As I described at f269559 commit, the package was not properly built because it did not have the list of dependencies when the whl was built.
from wfexs-backend.
jmfernandez
commented on August 18, 2024
Hi again (again),
I have been trying to reproduce what it is happening, and one hidden issue was that WfExS was not reporting the workflow mismatches, i.e. when an engine was raising exceptions due problems detecting a valid workflow. After the changes of commits 5191613 and e166d1e , now it is reporting the next (I'm including here a small relevant fragment):
2023-07-12 22:11:12,400 - [CWLWorkflowEngine _enrichWorkflowDeps 558][DEBUG] /home/jmfernandez/projects/WfExS-backend/workflow_examples/HUTCH/46/wfexs-backend-test/CWLWorkflowEngine/3.1.20230601100705 --print-deps =>
2023-07-12 22:11:12,401 - [wfexs_backend.workflow::WF fetchWorkflow 1174][ERROR] Engine CWL did not recognize the workflow as a valid one. Reason:
Traceback (most recent call last):
File "/home/jmfernandez/projects/WfExS-backend/wfexs_backend/workflow.py", line 1162, in fetchWorkflow
engineVer, candidateLocalWorkflow = engine.identifyWorkflow(
File "/home/jmfernandez/projects/WfExS-backend/wfexs_backend/cwl_engine.py", line 319, in identifyWorkflow
newLocalWf = self._enrichWorkflowDeps(newLocalWf, engineVer)
File "/home/jmfernandez/projects/WfExS-backend/wfexs_backend/cwl_engine.py", line 579, in _enrichWorkflowDeps
raise WorkflowEngineException(errstr)
wfexs_backend.engine.WorkflowEngineException: Could not get workflow dependencies running cwltool --print-deps from /home/jmfernandez/projects/WfExS-backend/workflow_examples/HUTCH/46/wfexs-backend-test_WorkDir/fc4796a7-f49e-439b-955a-abf337505cf6/workflow sec-hutchx86.cwl with /home/jmfernandez/projects/WfExS-backend/workflow_examples/HUTCH/46/wfexs-backend-test/CWLWorkflowEngine/3.1.20230601100705. Retval 1
======
STDOUT
======
======
STDERR
======
INFO /home/jmfernandez/projects/WfExS-backend/workflow_examples/HUTCH/46/wfexs-backend-test/CWLWorkflowEngine/3.1.20230601100705/bin/cwltool 3.1.20230601100705
INFO Resolved 'sec-hutchx86.cwl' to 'file:///home/jmfernandez/projects/WfExS-backend/workflow_examples/HUTCH/46/wfexs-backend-test_WorkDir/fc4796a7-f49e-439b-955a-abf337505cf6/workflow/sec-hutchx86.cwl'
ERROR Tool definition failed validation:
[Errno 2] No such file or directory: '/home/jmfernandez/projects/WfExS-backend/workflow_examples/HUTCH/46/wfexs-backend-test_WorkDir/fc4796a7-f49e-439b-955a-abf337505cf6/workflow/rquest-oneshotx86.cwl'
Indeed, the local workflow is depending on other cwl file. But you are telling WfExS that the workflow is only a single file, instead of giving it a "directory" and a starting point (i.e. a context). If you try something similar to the next:
# test-stage.yml
workflow_id: file:///root/hutch/workflows#subdirectory=sec-hutchx86.cwl
workflow_config:
container: 'docker'
secure: false
nickname: 'vas-workflow'
cacheDir: /tmp/wfexszn6siq2jtmpcache
crypt4gh:
key: cosifer_test1_cwl.wfex.stage.key
passphrase: mpel nite ified g
pub: cosifer_test1_cwl.wfex.stage.pub
outputs:
output_file:
c-l-a-s-s: File
glob: "output.json"
params:
body:
c-l-a-s-s: File
url:
- https://raw.githubusercontent.com/HDRUK/hutch/main/workflows/inputs/rquest-query.json
is_availability: true
db_host: "localhost"
db_name: "hutch"
db_user: "postgres"
db_password: "example"it should work (it's true, keyword subdirectory is a bit misleading, but it was named thinking on Nextflow scenarios).
from wfexs-backend.
Related Issues (20)
- Add support for compact `drs` identifiers HOT 1
- Add support for `secondaryFiles` CWL HOT 3
- Add support to `dav` and `davs` schemes
- Cannot download content from ftp HOT 5
- TypeError: Multiple inheritance with NamedTuple is not supported HOT 3
- Parsing of Nextflow DSL2 workflows HOT 1
- Add support for swh permanent identifiers HOT 1
- Add several checks in the code to detect containers unavailable for the current hardware architecture
- Consider supporting other processor architectures through multiarch/qemu-user-static HOT 1
- Use pyinvoke and Fabric
- Fix dependency on newer versions of python-dxf HOT 1
- Introduce codemeta.json
- Secrets/secret inputs HOT 2
- Allow using as a staging source a Workflow Run RO-Crate
- Bug in path resolution in local config file HOT 3
- PermissionError: [Errno 13] Permission denied: '/home/ansible/wfexs-backend-test/wf-cache' HOT 3
- Can't execute workflows using podman HOT 3
- `dot` dependency should be optional
- WfExS-backend init issues
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from wfexs-backend.