Git Product home page Git Product logo

hatchet-python's Introduction

Hatchet Python SDK

This is the Hatchet Python SDK. For usage, see the docs.

hatchet-python's People

Contributors

abelanger5 avatar grutt avatar steebchen avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

hatchet-python's Issues

enhancement: allow control over JSON serialization, nested_asyncio, and Context class

Hi, I would like to suggest a few enhancements to the sdk.

Version: 0.23.0

In my own workflow library, I've had to implement a few patches in order to overcome the existing behavior of the sdk.

JSON Serialization

If there was a way to allow for a custom JSON serialization/deserialization method, it would be extremely useful to handle the input, especially with object hooks. It's very easy to forget a custom datetime.datetime object, and if the ser/deser hook can handle those inputs, it would allow for more flexibility for the end-user.

I currently patch the behavior of the following:

  • hatchet_sdk.worker.Worker
  • hatchet_sdk.context.Context
  • hatchet_sdk.context.ChildWorkflowRef

nest_asyncio

My use case is that I would like to spawn multiple workflows, each with their own workers at the same time.

Due to nest_asyncio, this causes potential problems with downstream libraries that rely on the same EventLoop, which is a problem, depending on when hatchet_sdk is imported and nest_asyncio.apply() occurs.

- Task <Task pending name='Task-414' coro=<BaseComponent.ahandle_data_one_iter() running at .../types/component/base.py:482> cb=[_wait.<locals>._on_completion() at .../lib/python3.10/asyncio/tasks.py:475]> got Future <Future pending> attached to a different loop

It would be helpful if I was able to disable nest_asyncio.apply() with an env var, understanding the implications of doing so.

Context Class

It would be very useful to allow for the user to provide a custom Context class that gets used to create Context within workflow steps by the Worker. This would enable the user to add properties or methods.

Currently, I have it implemented as:

class Worker:
    """
    The Worker Object
    """
    def __init__(
        self,
        name: str,
        max_runs: int | None = None,
        debug: bool = False,
        handle_kill: bool = True,
        config: ClientConfig = None,
        context_class: Type[Context] = Context,
    ):
        ...
        self._context_cls = context_class


    def handle_start_step_run(self, action: Action):
        """
        Handles the start step run action
        """
        action_name = action.action_id
        context = self._context_cls(action, self.client)
       ...

    def handle_start_group_key_run(self, action: Action):
        """
        Handles the start group key run action
        """
        action_name = action.action_id
        context = self._context_cls(action, self.client)
        ....

Thanks!

bug: client.admin.run_workflow raises KeyError

SDK Version: 0.23.0

The latest raises an error due to the following:

class AdminClientImpl:
   ...
   def run_workflow(
        self,
        workflow_name: str,
        input: any,
        options: TriggerWorkflowOptions = None,
    ):
        """
        Handle the run workflow method
        """
        try:
            payload_data = json_serializer.dumps(input)

            try:
                meta = None if options is None else options["additional_metadata"]  # < raises KeyError Here
                # meta = None if options is None else options.get("additional_metadata") 
   

This was triggered by context.spawn_workflow

class Context:
   ...
    def spawn_workflow(self, workflow_name: str, input: dict = None, key: str = None):
        """
        Spawns a workflow
        """
        input = input or {}
        workflow_run_id = self.action.workflow_run_id
        step_run_id = self.action.step_run_id
        
        # Since `additional_metadata` doesn't exist here
        options: ScheduleTriggerWorkflowOptions = {
            "parent_id": workflow_run_id,
            "parent_step_run_id": step_run_id,
            "child_key": key,
            "child_index": self.spawn_index,
        }

        self.spawn_index += 1
        child_workflow_run_id = self.client.admin.run_workflow(
            workflow_name, input, options
        )

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.