The performance optimization that would make use of the advantages of the workflow definition, is to enable the scheduler to pre-warm functions. The controller will poke the Fission functions to warm them up before actually invoking them.

The process would look like this:

• Scheduler receives workflow, where task A is on the scheduling horizon and task B depends on task A.
• the scheduler schedules task A for execution
• The scheduler schedules task B for pre-warming

We need a service separate from the 'workflow engine' that only contains the api server logic, without any active controller. This will be useful with the integration in Fission, essentially creating a read-only fission workflows instance.

Currently the user needs to manually compile their yaml workflow definition to json using the wfparser cli tool.

The intention is have this abstracted away from the user by using the builder in Fission. This should allow users to provide the yaml definitions directly to Fission.

Pick either Python or Javascript, and write a set of classes and functions that allow a user to specify a workflow in that language, instead of YAML.

User should be able to define dynamic tasks (such as loops and other higher order functions) within a workflow definition, using Javascript or Go, and have them interpreted by the workflow engine as part of workflow execution.

To make dynamic task definition easy, the workflow engine should do most of the work of type checks etc. around dynamic task inputs and outputs. The user should have to write as little code as possible.

Both implementations (Javascript and Go) would involve the workflow engine loading the dynamic task and running it.

The Javascript implementation can use the existing Javascript interpreter, which already allows us to isolate functions as well as run them with timeouts. That will ensure that a misbehaved dynamic task from one workflow won't affect other workflows. If we solve #76 using Javascript, then that same set of primitives can be used within the dynamic task definition.

The Go implementation will need the builder to compile the Go tasks as one or more plugins. The workflow engine would load these as plugins and run them. It would have to do this in a separate process, for isolation; otherwise a misbehaving plugin could affect other workflows. There's a little more complexity in the Go implementation because of this out-of-process requirement. Also, the user would need to learn how to specify a workflow in Go using the workflow engine's type system, which is non-trivial because of some of the dynamic typing that the workflow engine does.

Overall, Javascript sounds like a better fit from a developer experience point of view, with the only advantage of the Go implementation being that you can probably get more performance out of it than the interpreted Javascript.

The current examples in the readme are old and 'abstract'. They should be updated to actually functional examples

Currently only internal functions can be dynamic tasks (tasks that return a task). Support should be added to the fission runtime environment to also allow fission functions to be dynamic tasks.

One approach would be that in the presence of a special header X-Fission-Workflow-Dynamic the body is interpreted as a task. Or, maybe even better, having the content-type be something like application/vnd+fission.workflow+json. By specifically requiring the header we avoid outputs to be accidentally being interpreted as dynamic tasks.

Performance of the workflow engine is key. So it should be measured accordingly. This issue tracks the progress of that effort.

Thoughts on it:

• Reuse metric decisions of Fission. So Prometheus is the way to go.
• Zipkin/opentracing support could be an option, although not needed, as the workflow engine has knowledge of the call graph anyway (except for implicit calls in functions)

Currently the JSON workflow definition representation is a direct translation from the internal protobuf workflow spec. Due to the limitations of Protobuf, TypedValues cannot be represented as arbitrary types in the JSON format, which leads to unreadable serialized values (see github.com/fission/fission-workflow/tree/master/examples/whales/maybewhale.wf.json)

We should separate the JSON format from the internal Protobuf data structure to ensure that the inputs remain readable to the user.

Documents:

• Terminology
• Architecture
• Interface / API / protocol
• High-level design decisions/concepts: workflows, invocations, events, controller, scheduler, proxy ect.
• Installation of/running example
• (prospective) use case / developer experience

Currently implicit expressions are recognized only when there is a trailing $. This lacks flexibility in the expressions. There should be ways to improve this identification of expression types vs. regular strings.

Already partly supported, but needs to be fleshed out.

We need a way to either manually or automatically clean up old workflow invocation data.

Similar to #64 we also might want to allow users to extend the functionality of the expression engine. Making it pluggable will most likely result in something similar to internal functions: a user uploads a golang plugin to an endpoint, the system validates it and makes it available to the expressions in the workflows.

Partial application should allow users to create workflows by just specifying a part of the arguments.

This would already be useful when giving demo's. For example, you want to provide a webhook for a slack command, but the workflow also requires apikeys and other configs to be set. You create a partial application of the workflow by providing all the apikeys. This leaves just the argument (in this case the formdata-body) to be provided by Slack.

Currently users can only specify a single task or a reference to another workflow in a dynamic task. To avoid having to write workflows for small, trivial or application-specific use cases, it should be possible to write a inline workflow, analogous to how lambda's are used in Java/python/etc.

Ideally the inline workflow should match the workflow definition as closely as possible.

• Incorporate Fission's FunctionRef into workflow engine

It seems that the workflow-env gets killed/terminated. Does get rebooted properly.

Currently workflows are defined using a backward dependencies. For example:

myTask:
  ...
  requires:
    otherDepTask1: ~
    otherDepTask2: ~

While provides more self-contained tasks, for simple use cases it might be more straightforward to allow for forward dependencies.

myTask:
  ...
  on-complete:
  - nextTask1
  - nextTask2

On the backend the dependencies should resolve similarly: a task waits for all dependencies, whether specified forward or backward, to resolve. dependencyAwait allows tasks to wait for only a part of their dependencies. However, with the constraint that individual tasks can only be invoked once, this means that you can only create discriminators not 'multiple instances' (see http://workflowpatterns.com/).

Similar to Openstack's mistral, we can support both. The user should be able to specify which type of dependency definition they want to use.

apiVersion: v1
dependencies: forward|backward
tasks:
  ...

The default, though open for discussion, would be backward.

Currently the workflow-engine, due to prototyping reasons, is a separate component within the fission ecosystem. The plan is to ensure that the workflow engine is just another environment.

The benefits of it being a environment:

• It will be there for users actually using it.
• No need for users to go out and deploy it manually. Just add the environment like any other and you are done.
• Make use of the existing functionality provided to environments. e.g. autoscaling.

To achieve this

• A option is needed in Fission that allows environments to specify the maximum number of instances needed and whether it can specialize multiple functions. Currently Fission always maintains at least 3 environment instances and allows only one specialization per instance.
• Add a fetcher to the system that corresponds, similar to the fetcher used in other environments.
• A specialize endpoint needs to be added to the Fission Proxy.
• Functionality should be added to Fission to allow for gRPC communication (again, based on an option of the environment). This will allow us to also merge the Fission Proxy into the existing gRPC server, removing the need for an additional server (as is the situation now)

The currently implemented event store was written with a limited knowledge of NATS streaming, abusing some of the features.

Currently the naieve setup consists of the following subject structure

• workflow.<id>: each subchannel contains one workflow.
• invocation.<id> : each subchannel contains the activities of one workflow invocation.
• _activity: NATS streaming doesn't support wildcard subscriptions as of the moment of writing (see nats-io/nats-streaming-server#340 for progress on the feature). In order to replicate the behavior of a wildcard subscription this channel is used to publish notifications of when something happens in one of the various workflow/invocation subchannels. The event store implementation has one 'metasubscriptions' on this channel that creates new subscribers when new subjects appear.

Issues with this setup:

• subjects/channels seem expensive. NATS streaming puts limits on the number of subscriptions and the number of subjects. Though these are artificial, and can be increased, and it remains unclear what happens if changed to a much higher number.
• subjects/channels cannot be deleted in a straightforward way. This means, even if invocations complete, they still remain in NATS as channels/subjects.
• Messages cannot be deleted either. Nor can certain channels/messages be marked as deletable. Currently NATS just starts deleting the oldest messages once it is full.

Possible solutions:

1. Move workflows out of the event store. As these are considered immutable when generated/parsed, they can be kept out of the event store and left to be handled by Fission.
   • Though it might be needed to store them somewhere persistent, as workflow invocations are tied to these parsed workflows. Information which might be lost if the parsed workflow is not stored.
2. Use a single subject for all the things. I am not sure what the performance hit of this would be as, subscribers would need to go over all messages when recreating the state of anything.
3. Use a subject per workflow. In this case the workflow is stored together with associated invocations. The problem here might be that you will not be able to delete (when that option becomes available) any subject as that would also
4. Keep the current implementation and work with the NATS team to implement some of the missing functionality:
   • More advanced garbage collection; ability to mark subjects/messages as GC'able
   • Wildcard subscription support for NATS streaming
   • Ability to delete or archive subjects (or even messages) manually
5. Switch to a different database or message bus. There is no perfect solution on the market yet that contains all the required properties of the event store (fast, lightweight, persistent, reliable, scalable). A partial implementation exists for BoltDB (dropped after realizing it would need implementation of the entire pubsub functionality) which might be an alternative.

Currently, for the prototype, this is a low priority issue, as for small usage (<1000 invocations) it works just fine. Nothing is persisted yet, as fission-nats.yml deployment is still using in-memory, and can be cleared by simply restarting that deployment. So, until the prototype is advanced enough that it becomes clear what is needed from the event store, the current implementation is okay.

Let's use this issue to keep track of workflow visualization. We also want to implement a GUI editor for workflows. There's enough in common between these two efforts that we can use the same issue for now.

This issue tracks the state of the incorporation of workflow management in the existing Fission CLI or into a separate CLI. It might just be that we decide to not integrate any thing into the client specifically to workflows.

(Proposed) API functionality:

• Workflow CRUD: managed by regular Fission CLI (fission wf == fission fn)
• List invocations per workflow. Potential info to show: ID, status, last update
• Get invocation status/logs

Roadmap:

• Decide on the functionality to be exposed in the CLI
• Implement a standalone CLI for the workflows
• Merge into Fission CLI

Currently internal functions (like if, noop and sleep) are hardcoded into the workflow engine, though they are built to be extensible. A user should be able to add internal functions during runtime, without the need to rebuild/redeploy the workflow engine. Probably requires a separate endpoint that allows loading (http or shared volume) of Golang plugins.

There used to be (in the JSON format) support to force the type of an input. This functionality should also be supported in the YAML format.

Users shouldn't have to learn and install two separate CLIs.

Given that a major emphasis is made on re-using workflows and recursion over iteration. It is vital for the workflow engine to have some sort of recursion detection.

Solution 1
A "parent" parameter could be added (as a header for example) that links a workflow to its parent workflow. From there the workflow engine can analyze the workflow dependency graph to determine whether a infinite recursion is occurring

Potential issues:

• Requires centralized state (of the dependency graph) when multiple workflow engine instances are running.
• Compute intensive

Solution 2
Add a stack parameter to the request, containing all preceding workflow invocation ids. The workflow engine can infer can simply call a stack overflow if the stack becomes to large.

Potential issues:

• Depending on the stack limit, the request size may become a significant overhead to the request.

...

Feature request: allow single-stepping through workflows while being able to inspect task inputs and outputs.

Sometimes when a workflow is invoked once, some of the tasks are invoked more than once.

(Sorry, I don't have more info right now.)

This is a aggregate issue of all minor TODOs not worth a separate issue. Noted here mainly for my own organization

• Add README.md to simple-example
• Move Swagger API out of Docs folder (doesn't belong there)
• Move parser responsiblity out of the create_workflow api command. Currently it is not retried if it fails or periodically to update the workflow. Combined with the plan to provide 'version policies' it might need to be moved to the controller.
• Implement control loop in controller
• Refactor scheduler to a more structure format (using GOSTA framework for example)
• Fix the projection component (currently a lot of duplication, lacking a general interface)
• (Nats) Unsubscribe from completed invocations.
• Evict completed invocations from cache (or just when capacity is met)
• introduce overflow flag in the subscription model. If set, increase frequency of control loop that checks the projector for active invocations
• Generalize subscription model (currently specific to wf invocations)
• Capture errors generated by fission functions.

The fission controller proxy for the workflows apiserver doesn't work if workflows is installed in a different namespace from the fission controller.

$ wfcli status
panic: unknown error (status 502): {resp:0x10a78190} 

$ kubectl -n fission logs controller...
2017/10/25 15:53:42 http: proxy error: dial tcp: lookup workflows-apiserver on 10.43.240.10:53: no such host
127.0.0.1 - - [25/Oct/2017:15:53:42 +0000] "GET /proxy/workflows-apiserver/status HTTP/1.1" 502 0

To fix this, the proxy must either discover the workflows installation somehow (maybe through a configmap) or we should require workflows to be installed in the same namespace as fission (not even sure that's possible given that helm allows the user to override the installation namespace).

In order to account for some of the requirements of this project: performance, scalability, reliability. A simple benchmark is needed. As the workflow engine is not different from any other Fission function, this benchmark tool should easily be generalizable for any Fission function (some prior work of before my internship exists for this)

Functionality:

• Given some workflow/function definitions, automatically upload them to Fission
• Given some parameters (or even trace data) execute invoke workflows.
• Record (user-level) statistics.

The env create command has the wrong image tag (it is 0.3.0 not v0.3.0):

The right command is:

fission env create --name binary --image fission/binary-env:0.3.0

but the NOTES contains the command with an extra 'v' in the tag.

Currently there is no check, and a workflow will basically timeout after getting stuck. Due to the static declaration of dependencies between tasks, within the workflow the validation should be trivial.

I have a very simple workflow like:

apiVersion: 1
description: Collect a form - computes a risk then sends SMS with result
output: CreateResult
tasks:
  # Fetch incoming data and update status. Computes risk.
  SetToPending:
    run: set-to-pending
    inputs: "{$.Invocation.Inputs.default}"

  # Adds a status depending on value of the risk
  MakeDecision:
    run: make-decision
    inputs: "{$.Tasks.SetToPending.Output}"
    requires:
    - SetToPending

  SendSMS:
    run: if
    inputs:
      if: "{$.Tasks.SetToPending.Output.risk > 70}"
      then: 
        run: send-sms
        inputs: "{$.Tasks.MakeDecision.Output}"
      else: 
        run: send-sms
        inputs: "{$.Tasks.MakeDecision.Output}"
    requires:
    - SetToPending
    - MakeDecision

  CreateResult:
    run: compose
    inputs:
      id: "{$.Tasks.SetToPending.Output.id}"
      status: "{$.Tasks.MakeDecision.Output.status}"
      risk: "{$.Tasks.MakeDecision.Output.risk}"
      sid: "{$.Tasks.SendSMS.Output.sid}"
    requires:
    - SetToPending
    - MakeDecision
    - SendSMS

the inputs are supposed to be JSON. I made a mistake calling the function and my JSON was malformed. Then the workflow engine goes crazy and retries for ever:

$ stern workflow --namespace fission-function
"2017-10-20T08:47:17Z" level=info msg="Invoking function 'if' for task 'SendSMS'" 
"2017-10-20T08:47:17Z" level=info msg="Resolved expression" key=then resolved=type:"flow" value:"\n\004then\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  val=type:"flow" value:"\n\004then\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  
"2017-10-20T08:47:17Z" level=info msg="Resolved expression" key=else resolved=type:"flow" value:"\n\004else\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  val=type:"flow" value:"\n\004else\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  
"2017-10-20T08:47:17Z" level=error msg="Failed to parse input: TypeError: Cannot access member 'risk' of undefined" inputKey=if val=type:"expr" value:"{$.Tasks.SetToPending.Output.risk > 70}"  
"2017-10-20T08:47:17Z" level=error msg="WorkflowInvocation action failed" action=&{0x19fffbf0 0x1a122670 0x1a123210 0x1a33cba0 0x1a2e9050} 
"2017-10-20T08:47:18Z" level=info msg="Scheduler evaluating..." invoke=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d workflow=74926bd6-b4e5-11e7-8ff6-42010a9a010b 
"2017-10-20T08:47:18Z" level=info msg="Checking if dependencies have been satisfied" completedDeps=2 max=2 task=SendSMS 
"2017-10-20T08:47:18Z" level=info msg="task found on horizon" task=SendSMS 
"2017-10-20T08:47:18Z" level=info msg="Checking if dependencies have been satisfied" completedDeps=3 max=3 task=CreateResult 
"2017-10-20T08:47:18Z" level=info msg="Determined schedule" invocation=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d invoke=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d schedule=1 workflow=74926bd6-b4e5-11e7-8ff6-42010a9a010b 
"2017-10-20T08:47:18Z" level=info msg="Invoking function 'if' for task 'SendSMS'" 
"2017-10-20T08:47:18Z" level=info msg="Resolved expression" key=then resolved=type:"flow" value:"\n\004then\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  val=type:"flow" value:"\n\004then\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  
"2017-10-20T08:47:18Z" level=info msg="Resolved expression" key=else resolved=type:"flow" value:"\n\004else\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  val=type:"flow" value:"\n\004else\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  
"2017-10-20T08:47:18Z" level=error msg="Failed to parse input: TypeError: Cannot access member 'risk' of undefined" inputKey=if val=type:"expr" value:"{$.Tasks.SetToPending.Output.risk > 70}"  
"2017-10-20T08:47:18Z" level=error msg="WorkflowInvocation action failed" action=&{0x19fffbf0 0x1a122670 0x1a123210 0x1a33cba0 0x1a41aa70} 
"2017-10-20T08:47:19Z" level=info msg="Scheduler evaluating..." invoke=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d workflow=74926bd6-b4e5-11e7-8ff6-42010a9a010b 
"2017-10-20T08:47:19Z" level=info msg="Checking if dependencies have been satisfied" completedDeps=3 max=3 task=CreateResult 
"2017-10-20T08:47:19Z" level=info msg="Checking if dependencies have been satisfied" completedDeps=2 max=2 task=SendSMS 
"2017-10-20T08:47:19Z" level=info msg="task found on horizon" task=SendSMS 
"2017-10-20T08:47:19Z" level=info msg="Determined schedule" invocation=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d invoke=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d schedule=1 workflow=74926bd6-b4e5-11e7-8ff6-42010a9a010b 
"2017-10-20T08:47:19Z" level=info msg="Invoking function 'if' for task 'SendSMS'" 
"2017-10-20T08:47:19Z" level=info msg="Resolved expression" key=then resolved=type:"flow" value:"\n\004then\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  val=type:"flow" value:"\n\004then\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  
"2017-10-20T08:47:19Z" level=info msg="Resolved expression" key=else resolved=type:"flow" value:"\n\004else\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  val=type:"flow" value:"\n\004else\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  
"2017-10-20T08:47:19Z" level=error msg="Failed to parse input: TypeError: Cannot access member 'risk' of undefined" inputKey=if val=type:"expr" value:"{$.Tasks.SetToPending.Output.risk > 70}"  
"2017-10-20T08:47:19Z" level=error msg="WorkflowInvocation action failed" action=&{0x19fffbf0 0x1a122670 0x1a123210 0x1a33cba0 0x1a326120} 
"2017-10-20T08:47:20Z" level=info msg="Scheduler evaluating..." invoke=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d workflow=74926bd6-b4e5-11e7-8ff6-42010a9a010b 
"2017-10-20T08:47:20Z" level=info msg="Checking if dependencies have been satisfied" completedDeps=3 max=3 task=CreateResult 
"2017-10-20T08:47:20Z" level=info msg="Checking if dependencies have been satisfied" completedDeps=2 max=2 task=SendSMS 
"2017-10-20T08:47:20Z" level=info msg="task found on horizon" task=SendSMS 
"2017-10-20T08:47:20Z" level=info msg="Determined schedule" invocation=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d invoke=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d schedule=1 workflow=74926bd6-b4e5-11e7-8ff6-42010a9a010b 
"2017-10-20T08:47:20Z" level=info msg="Invoking function 'if' for task 'SendSMS'" 
"2017-10-20T08:47:20Z" level=info msg="Resolved expression" key=then resolved=type:"flow" value:"\n\004then\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  val=type:"flow" value:"\n\004then\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  
"2017-10-20T08:47:20Z" level=info msg="Resolved expression" key=else resolved=type:"flow" value:"\n\004else\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  val=type:"flow" value:"\n\004else\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  
"2017-10-20T08:47:20Z" level=error msg="Failed to parse input: TypeError: Cannot access member 'risk' of undefined" inputKey=if val=type:"expr" value:"{$.Tasks.SetToPending.Output.risk > 70}"  
"2017-10-20T08:47:20Z" level=error msg="WorkflowInvocation action failed" action=&{0x19fffbf0 0x1a122670 0x1a123210 0x1a33cba0 0x1a098310} 
"2017-10-20T08:47:21Z" level=info msg="Scheduler evaluating..." invoke=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d workflow=74926bd6-b4e5-11e7-8ff6-42010a9a010b 
"2017-10-20T08:47:21Z" level=info msg="Checking if dependencies have been satisfied" completedDeps=2 max=2 task=SendSMS 
"2017-10-20T08:47:21Z" level=info msg="task found on horizon" task=SendSMS 
"2017-10-20T08:47:21Z" level=info msg="Checking if dependencies have been satisfied" completedDeps=3 max=3 task=CreateResult 
"2017-10-20T08:47:21Z" level=info msg="Determined schedule" invocation=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d invoke=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d schedule=1 workflow=74926bd6-b4e5-11e7-8ff6-42010a9a010b 
"2017-10-20T08:47:21Z" level=info msg="Invoking function 'if' for task 'SendSMS'" 
"2017-10-20T08:47:21Z" level=info msg="Resolved expression" key=then resolved=type:"flow" value:"\n\004then\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  val=type:"flow" value:"\n\004then\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  
"2017-10-20T08:47:21Z" level=info msg="Resolved expression" key=else resolved=type:"flow" value:"\n\004else\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  val=type:"flow" value:"\n\004else\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  
"2017-10-20T08:47:21Z" level=error msg="Failed to parse input: TypeError: Cannot access member 'risk' of undefined" inputKey=if val=type:"expr" value:"{$.Tasks.SetToPending.Output.risk > 70}"  
"2017-10-20T08:47:21Z" level=error msg="WorkflowInvocation action failed" action=&{0x19fffbf0 0x1a122670 0x1a123210 0x1a33cba0 0x1a3bdb00} 
"2017-10-20T08:47:22Z" level=info msg="Scheduler evaluating..." invoke=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d workflow=74926bd6-b4e5-11e7-8ff6-42010a9a010b 
"2017-10-20T08:47:22Z" level=info msg="Checking if dependencies have been satisfied" completedDeps=3 max=3 task=CreateResult 
"2017-10-20T08:47:22Z" level=info msg="Checking if dependencies have been satisfied" completedDeps=2 max=2 task=SendSMS 
"2017-10-20T08:47:22Z" level=info msg="task found on horizon" task=SendSMS 
"2017-10-20T08:47:22Z" level=info msg="Determined schedule" invocation=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d invoke=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d schedule=1 workflow=74926bd6-b4e5-11e7-8ff6-42010a9a010b 
"2017-10-20T08:47:22Z" level=info msg="Invoking function 'if' for task 'SendSMS'" 
"2017-10-20T08:47:22Z" level=error msg="Failed to parse input: TypeError: Cannot access member 'risk' of undefined" inputKey=if val=type:"expr" value:"{$.Tasks.SetToPending.Output.risk > 70}"  
"2017-10-20T08:47:22Z" level=error msg="WorkflowInvocation action failed" action=&{0x19fffbf0 0x1a122670 0x1a123210 0x1a33cba0 0x1b678bf0} 
"2017-10-20T08:47:23Z" level=info msg="Scheduler evaluating..." invoke=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d workflow=74926bd6-b4e5-11e7-8ff6-42010a9a010b 
"2017-10-20T08:47:23Z" level=info msg="Checking if dependencies have been satisfied" completedDeps=3 max=3 task=CreateResult 
"2017-10-20T08:47:23Z" level=info msg="Checking if dependencies have been satisfied" completedDeps=2 max=2 task=SendSMS 
"2017-10-20T08:47:23Z" level=info msg="task found on horizon" task=SendSMS 
"2017-10-20T08:47:23Z" level=info msg="Determined schedule" invocation=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d invoke=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d schedule=1 workflow=74926bd6-b4e5-11e7-8ff6-42010a9a010b 
"2017-10-20T08:47:23Z" level=info msg="Invoking function 'if' for task 'SendSMS'" 
"2017-10-20T08:47:23Z" level=info msg="Resolved expression" key=then resolved=type:"flow" value:"\n\004then\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  val=type:"flow" value:"\n\004then\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  
"2017-10-20T08:47:23Z" level=info msg="Resolved expression" key=else resolved=type:"flow" value:"\n\004else\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  val=type:"flow" value:"\n\004else\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  
"2017-10-20T08:47:23Z" level=error msg="Failed to parse input: TypeError: Cannot access member 'risk' of undefined" inputKey=if val=type:"expr" value:"{$.Tasks.SetToPending.Output.risk > 70}"  
"2017-10-20T08:47:23Z" level=error msg="WorkflowInvocation action failed" action=&{0x19fffbf0 0x1a122670 0x1a123210 0x1a33cba0 0x1a09e090} 
"2017-10-20T08:47:24Z" level=info msg="Scheduler evaluating..." invoke=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d workflow=74926bd6-b4e5-11e7-8ff6-42010a9a010b 
"2017-10-20T08:47:24Z" level=info msg="Checking if dependencies have been satisfied" completedDeps=2 max=2 task=SendSMS 
"2017-10-20T08:47:24Z" level=info msg="task found on horizon" task=SendSMS 
"2017-10-20T08:47:24Z" level=info msg="Checking if dependencies have been satisfied" completedDeps=3 max=3 task=CreateResult 
"2017-10-20T08:47:24Z" level=info msg="Determined schedule" invocation=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d invoke=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d schedule=1 workflow=74926bd6-b4e5-11e7-8ff6-42010a9a010b 
"2017-10-20T08:47:24Z" level=info msg="Invoking function 'if' for task 'SendSMS'" 
"2017-10-20T08:47:24Z" level=info msg="Resolved expression" key=then resolved=type:"flow" value:"\n\004then\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  val=type:"flow" value:"\n\004then\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  
"2017-10-20T08:47:24Z" level=info msg="Resolved expression" key=else resolved=type:"flow" value:"\n\004else\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  val=type:"flow" value:"\n\004else\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  
"2017-10-20T08:47:24Z" level=error msg="Failed to parse input: TypeError: Cannot access member 'risk' of undefined" inputKey=if val=type:"expr" value:"{$.Tasks.SetToPending.Output.risk > 70}"  
"2017-10-20T08:47:24Z" level=error msg="WorkflowInvocation action failed" action=&{0x19fffbf0 0x1a122670 0x1a123210 0x1a33cba0 0x1a098160} 
"2017-10-20T08:47:25Z" level=info msg="Scheduler evaluating..." invoke=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d workflow=74926bd6-b4e5-11e7-8ff6-42010a9a010b 
"2017-10-20T08:47:25Z" level=info msg="Checking if dependencies have been satisfied" completedDeps=2 max=2 task=SendSMS 
"2017-10-20T08:47:25Z" level=info msg="task found on horizon" task=SendSMS 
"2017-10-20T08:47:25Z" level=info msg="Checking if dependencies have been satisfied" completedDeps=3 max=3 task=CreateResult 
"2017-10-20T08:47:25Z" level=info msg="Determined schedule" invocation=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d invoke=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d schedule=1 workflow=74926bd6-b4e5-11e7-8ff6-42010a9a010b 
"2017-10-20T08:47:25Z" level=info msg="Invoking function 'if' for task 'SendSMS'" 
"2017-10-20T08:47:25Z" level=info msg="Resolved expression" key=then resolved=type:"flow" value:"\n\004then\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  val=type:"flow" value:"\n\004then\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  
"2017-10-20T08:47:25Z" level=info msg="Resolved expression" key=else resolved=type:"flow" value:"\n\004else\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  val=type:"flow" value:"\n\004else\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  
"2017-10-20T08:47:25Z" level=error msg="Failed to parse input: TypeError: Cannot access member 'risk' of undefined" inputKey=if val=type:"expr" value:"{$.Tasks.SetToPending.Output.risk > 70}"  
"2017-10-20T08:47:25Z" level=error msg="WorkflowInvocation action failed" action=&{0x19fffbf0 0x1a122670 0x1a123210 0x1a33cba0 0x1a3d7ef0} 
"2017-10-20T08:47:26Z" level=info msg="Scheduler evaluating..." invoke=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d workflow=74926bd6-b4e5-11e7-8ff6-42010a9a010b 
"2017-10-20T08:47:26Z" level=info msg="Checking if dependencies have been satisfied" completedDeps=3 max=3 task=CreateResult 
"2017-10-20T08:47:26Z" level=info msg="Checking if dependencies have been satisfied" completedDeps=2 max=2 task=SendSMS 
"2017-10-20T08:47:26Z" level=info msg="task found on horizon" task=SendSMS 
"2017-10-20T08:47:26Z" level=info msg="Determined schedule" invocation=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d invoke=wi-da56801c-3125-4a21-bbf7-dd2a9563f52d schedule=1 workflow=74926bd6-b4e5-11e7-8ff6-42010a9a010b 
"2017-10-20T08:47:26Z" level=info msg="Invoking function 'if' for task 'SendSMS'" 
"2017-10-20T08:47:26Z" level=info msg="Resolved expression" key=then resolved=type:"flow" value:"\n\004then\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  val=type:"flow" value:"\n\004then\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  
"2017-10-20T08:47:26Z" level=info msg="Resolved expression" key=else resolved=type:"flow" value:"\n\004else\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  val=type:"flow" value:"\n\004else\032\010send-sms\"0\n\007default\022%\n\004expr\022\035{$.Tasks.MakeDecision.Output}0\000"  
"2017-10-20T08:47:26Z" level=error msg="Failed to parse input: TypeError: Cannot access member 'risk' of undefined" inputKey=if val=type:"expr" value:"{$.Tasks.SetToPending.Output.risk > 70}"  
"2017-10-20T08:47:26Z" level=error msg="WorkflowInvocation action failed" action=&{0x19fffbf0 0x1a122670 0x1a123210 0x1a33cba0 0x1a3f1df0} 

I could not find a way to stop that. Currently the only option I found is to do

1. Delete the NATS deployment and recreate it
2. Delete the workflow pod in the fission-function namespace

Is there another and better way?

Thanks!

Currently FW uses the poolmgr and controller to resolve and invoking. This was a temporary solution as the router lacked the capability to route to a specific function, based on UID rather than name.

It makes more sense to use the router, because we avoid duplication in things such as caching. However we need to explore/think a bit about how the router fits into this, and what still needs to be changed

@soamvasani fyi

Does this project really need to use Bazel?

The output field can be used to manipulate or override the output of a task. This would prevent the need of an additional transformation task or tasks having to manipulate it in their inputs.

The actual output of a task is calculated by taking the actual function output and applying the output on to it.

Once it is merged into Fission the builder functionality should be reused to translate YAML to the JSON workflow format. This allows us to keep the workflow engine from having to deal with other formats other than the JSON format.

Currently only the default input is mapped to the body of fission functions.

To support passing inputs to the headers and query of the fission function invocation, we need to add the following mappings in the fission runtime environment.

Inputs:

• headers : JSON object (map) mapped to HTTP headers
• query : JSON object (map) mapped to query parameters.

Optionally, each entry in the headers and queries could also be map to support multiple headers/query params with the same key, but that is not a terrible thing not to support.

@soamvasani

Currently the short control loop that loops over the cache in the controller. However, in (edge) cases the invocation might no longer be in the cache, but still need to be handled by the controller. The long control loop (as described in the docs) will run less frequent, but willbe more thorough in finding relevant workflow invocations.

Currently the values are mapped to and from HTTP requests/responses using the content-type header implicitly. Although it currently works out okay for the current use cases, it might not work for things like binary data. Improvement are needed to have a more explicit mapping between typedvalues and http requests/responses.

Mention the following:

• General concept: javascript expressions
• data structures: $, task, output
• Utility functions: uid(), input(task, <key>), output(task), param(<key>)

Most of the workflow engines contain some sort of a visualizer to show the execution of a workflow, which improves the monitor-ability and debug-ability of workflows.

The UI can be divided in two parts:

• visualizer : showing various aspects of workflows and their invocations.
• editor : a GUI for building workflows (similar to NodeRED)

The editor is currently considered out of scope for the coming months, but the visualizer is definitely an important part of the workflow experience (especially for demonstrations).

Potential visualizer features:

• View Workflow(s)
• View statistics of workflow (e.g. % succeeded invocations, invocation latency percentiles, etc.)
• List Workflow Invocations per workflow
• View Workflow invocation (state, logs, decisions)
• Visualization of workflow (invocation)
• (optional) view workflow invocation live.
• (optional) add some of the cli functionality (invoke, cancel invocations), simple text editor for workflow

As discussed with @soamvasani

• Task/Function inconsistency becomes Task
• IfFunction: condition-consequent-alternative becomes if-then-else
• Task.Name becomes Task.FunctionRef
• Task.Dependencies becomes Task.Requires
• Funktor/control flow struct becomes dynamic task

From the start the internal functions have been designed to be fully pluggable. This means that the user should be able to dynamically add internal functions. Due to prototyping and time constraints this functionality has not made it in yet.

The solution will probable need a HTTP/gRPC endpoint to which users can upload a golang plugin. The system validates it and makes it available to the workflows

Currently the default pull policy of the environments are "IfNotPresent" and it is not possible to change it without manually updating the deployments.

It would be nice to be able to update the env with a --force which would trigger a forced update even if running on "latest"

This is not a big deal, especially with packaging coming up, just a nice to have.