Comments (8)
After some discussion and investigation with @jamesdreid I was able to identify the items()
behaviour has changed between yaql v1.1.3 and yaql v2.0.0
YAQL is currently pinned as yaql>=1.1.0
According to pypi, yaql was updated to 2.0.0
on 12 Jul 2021, which is just after st2 v3.5 and would have been included in the st2 v3.6
from orquesta.
Sorry, the issue was first identified in our workflows after an upgrade from 3.4.1 to 3.6. I do not know if it was there in 3.5 as we jumped straight to 3.6 on the system where these failing workflows were running.
from orquesta.
@nzlosh Any next steps? Do we need to pin yaql
in the requirements to fix it?
from orquesta.
I've looked into this further, this is the commit for yaql v2.0.0
The abstract base classes previously defined in 'collections' were moved
to 'collections.abc' in 3.3. The aliases will be removed in 3.10.- return isinstance(obj, collections.Iterator) + return isinstance(obj, collections.abc.Iterator)
If we pin to 1.1.3, builds will fail against py3.10 (which will be required to support ubuntu 22.04 and drop ubuntu 18.04). I think we're at the point where we'll need to fix the orquesta code. Which will be related to how it's detecting/handling collections.
from orquesta.
Looks like this PR may have been related to the attempted fix for a similar issue that may need some updates. #191
from orquesta.
Plus it appears that Orquesta may be importing "collections" as well and may need to be updated in a manner similar to the YAQL lib.
https://github.com/userlocalhost/orquesta/blob/4bc557574042a970a2101d15fecf626d0f93a4e7/orquesta/expressions/functions/workflow.py#L15
from orquesta.
Agreed. Let's fix orquesta, not the pin.
from orquesta.
It seems #191 is a partial fix. Testing a few expressions shows the following:
YAQL list ✔️
action: core.local
with: <% [1,2] %>
input:
cmd: echo <% item() %>
YAQL dict keys ❌
action: core.local
with: <% { "a" => 1, "b" => 2 }.keys() %>
input:
cmd: echo <% item() %>
'TypeError: The value of "<% { "a" => 1, "b" => 2 }.keys() %>" is not type of list.'
YAQL dict keys using X in format ❌
action: core.local
with: keyname in <% { "a" => 1, "b" => 2 }.keys() %>
input:
cmd: echo <% item() %>
'TypeError: The value of "<% { "a" => 1, "b" => 2 }.keys() %>" is not type of list.'
YAQL cast dict keys to list ✔️
action: core.local
with: <% { "a" => 1, "b" => 2 }.keys().toList() %>
input:
cmd: echo <% item() %>
Jinja list ❌
action: core.local
with: {{ [1,2] }}
input:
cmd: echo <% item() %>
"Failed to load workflow definition because while constructing a mapping
in "<unicode string>", line 7, column 12
found unacceptable key (unhashable type: 'list')
in "<unicode string>", line 7, column 14."
Jinja list using X in format ✔️
action: core.local
with: keyname in {{ [1,2] }}
input:
cmd: echo <% item() %>
stdout: '{keyname: 1}'
Jinja dict keys ❌
action: core.local
with: {{ {"a": 1, "b": 2}.keys() }}
input:
cmd: echo <% item() %>
"Failed to load workflow definition because while parsing a flow mapping
in "<unicode string>", line 7, column 12
did not find expected ',' or '}'
in "<unicode string>", line 7, column 30."
Jinja dict keys using X in format ❌
action: core.local
with: keyname in {{ {"a": 1, "b": 2}.keys() }}
input:
cmd: echo <% item() %>
"Failed to load workflow definition because mapping values are not allowed in this context
in "<unicode string>", line 7, column 29."
Jina cast dict keys to list using X in format ❌
action: core.local
with: keyname in {{ {"a": 1, "b": 2}.keys() | list }}
input:
cmd: echo <% item() %>
"Failed to load workflow definition because mapping values are not allowed in this context
in "<unicode string>", line 7, column 29."
It's not immediately obvious to me where the root cause of this issue is.
from orquesta.
Related Issues (20)
- Workflow join is not triggered on complete for failed task(s) HOT 8
- The ujson 2.0.x doesn't compatible with Orquesta HOT 1
- Retries using with-items runs a retry even on objects that succeeded as well HOT 5
- Add ability in task spec to wait for a lock before proceeding HOT 4
- Join failure within nested workflows can cause Parent workflow to run indefinitely.
- Join ALL and conditional branches conflict? HOT 6
- Incomplete next staged concurrent task with items if last running nested item fails.
- KeyValue DataStore does not load in Workflow HOT 2
- 'Inspect the workflow spec' and 'Instantiate the workflow conductor' take too long HOT 7
- Workflow join is not properly working if one step fails HOT 4
- Disable action notify triggers when action is executed under workflow context HOT 3
- Naming a workflow task "get_task" causes "'TaskSpec' object is not callable" error
- JOIN all with condition HOT 2
- Investigate using `rustworkx` instead of `networkx` HOT 1
- Workflow stuck with concurrency value of 0.
- Orquesta workflow inquiries responder user ID not found
- task with join: all starts without waiting for all the previous task completed when there is a loop
- Vague error HOT 1
- With Items Causing Workflow Tests to Fail When Items are an Empty List
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from orquesta.