Git Product home page Git Product logo

taskcontrols / py-taskcontrol Goto Github PK

View Code? Open in Web Editor NEW
3.0 4.0 2.0 1.21 MB

Control, Create, Run named workflows & tasks with before/after middlewares support, data persistence, orm, authentication, logging, multi-threading/processing, events, network sockets, webhooks, client-agent libraries, etc with plugins support [Actively Developed, Funding Invited] - pip3 install taskcontrol https://pypi.org/project/taskcontrol

Home Page: https://taskcontrols.github.io/

License: MIT License

Python 100.00%
python python3 python2 devops devops-tools task-runner task-queue workflow-automation workflow-tool azure-devops

py-taskcontrol's Introduction

taskcontrol (py-taskcontrol)

IN ACTIVE DEVELOPMENT

Workflow Automation Library with support for Concurrent or Event based processes or activities in Local/Network Automation Tasks, including CI/CD activities.
  • taskcontrol (py-taskcontrol) is a python library to create tasks in and based on named taskflow controls. It allows middlewares before and after each task with support for concurrent processing. taskcontrol can run single or multiple tasks during task invocation/runs.
  • It provides a simple decorator called task that takes arguments to set up the named taskflow controls. It also provides methods to create a plugin and allow working with tasks as a module and/or pre-created ordered task list. Taskcontrol allows for scaling of plugin development with various utilities like authentication, logging, concurrency, sockets, events, publisher-subscriber architectures, webhooks, client-server http api servers etc.

[Actively Developed, Funding Invited]

Features

  • Create Named task controls (tasks) - instance isolated and shared
  • Allows creation of before / after middlewares for each task with access read-only contexts and results of middlewares/tasks
  • Allows merging two instances of taskcontrols with namespace clash handling
  • Run a private instance, shared instance, and mix (private and shared) of instances (individual or all groups)
  • Allows working with Logging, Sockets, Events, Queues, etc
  • Allows working with Publisher-Subscriber (Message Queue) Architecture, Client-Agent Architecture, Agent-less Architecture, and Webhooks
  • In-Development:
    • Allows support for / working with Concurrency (Threads/ Processes) (T)
    • Allows working with Commands & Scripts (T), SSH (T), etc
    • Allows working with Scheduling (T), Files (T - normal, yaml, ini, and csv), etc
    • Allows working with ORMs/Databases (with in-memory support) (T), Authentication (T)
    • Allows creating, registering, and using tasks / workflows as a plugin
    • Planned Integrations with Subversioning, Build Tools, Deployment
    • Planned Integrations with Data Transformation / Analytics Tooling
    • Planned Integrations with Testing, and Infrastructure Automation toolings
    • Monitoring Support (T)
    • Allows working with best practices like Dependency Injection (T) within the library (including Tasks, Workflow)
    • Hooks support after dependency-injection package integration
    • Provided in and Allows plugins support for Python, Javascript languages

Features

Installation

Command:
  • Python

      pip3 install taskcontrol
    
Version:
In Development Version: 1.3.0b2 (functional - production ready with plugin and concurrency support, with demos)
Current Version: 1.3.0b1 (functional - production ready with plugin and concurrency support, with demos)
Previous Version: 1.3.0b0 (functional - production ready with most planned features with MVP, with demos)
Previous Version: 1.2.5 (functional - production ready with most planned features with MVP, with demos)
Previous Version: 1.1.2 (functional - production ready minor issues)
Previous Version: 1.1.0/1.1.1 (functional - not production ready - minor bug. Please upgrade to v1.1.2)

Releases Lists in short the releases of the package

Package Link:
https://github.com/taskcontrols/py-taskcontrol
https://pypi.org/project/taskcontrol
Website:
https://taskcontrols.github.io/

Supported Architectures within the Library

Workflow Task Execution Architecture

Workflow Execution Architecture

Workflow Task Execution Architecture

Publisher Subscriber, Message Queue, and Two Way Socket Events Architecture

Message Queue Architecture

Publisher Subscriber Architecture

Two Way Socket Events Architecture

Server - Client Agentless SSH Architecture

Agentless Architecture

Server - Client Agent Architecture

Server-Client Agent Architecture

REST API Architecture

REST API Access Architecture

Webhooks Architecture

Webhooks Architecture

Authentication and Data Persistence Architecture

Authentication and Data Persistence Architecture

Technical Specifications

Requirements:
  • Python 3.x and above (any OS)
Package Dependencies:
  • dependency-injector (planned todo)
Quick Demo:

General demo example - main_workflow.py

from taskcontrol import Workflow, task

sparrow = Workflow()

@task(
    name="migrate",
    task_instance=sparrow
)
def fly(ctx, result, *args, **kwargs):
    print("Running my task function: fly", args, kwargs)
    return args, kwargs

result = sparrow.start()
print(result)
Note:

Though it may support Python version 2.x. However, it has not been tested in 2.x. The Syntax and Features of the library supports Python version 2.x. Use at your own risk.

Wiki

Crazy Hint:

You can also create a simple workflow without taskcontrol using a simple list or nested list and loop through them using a for/while loop and invoke them during looping

# Loop the lists below and invoke the functions 
lst = ["f1", "f2", "f3"]
nest_lst = [["f1", "f2"], "f3", "f4", ["f5"]]


# Use a reducer if you want to send args to next function like below
def test(a,b):
    print(a,b)
    return {"a":a, "b":b}
def tester(a,b):
    print(a,b)
    return None

kwargs_for_first_function_the_its_returns_or_other_value_for_next_func = {"a":"a", "b":"b"}
ls = [kwargs_for_first_function_the_its_returns_or_other_value_for_next_func, test, tester]
import functools 
def red(kwargs_for_first_then_func, p):
    i = p(kwargs.get("a"), kwargs.get("b"))
    return i
functools.reduce(red, ls)

Status

  • In Active Development (taskcontrol version 1.3.0 and Common Integrations and Activities as Plugins)

Support

Paypal.me/taskcontrols

Be a Patreon

License

The MIT License (MIT) - See LICENSE for further details

Copyright © 2020 - till library works

[email protected]

py-taskcontrol's People

Contributors

ganeshkbhat avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

ganeshkbhat

py-taskcontrol's Issues

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.