Git Product home page Git Product logo

talepy's Introduction

Tale Distributed Transactions

Build Status

What?

Tale is a small library to help write a "distributed transaction like" object across a number of services. It's loosely based on the saga pattern. A good intro is available on the couchbase blog: https://blog.couchbase.com/saga-pattern-implement-business-transactions-using-microservices-part/

Installation

pipenv install talepy

Example Usage

An example use case of this would be some holiday booking software broken down into a few services.

Assuming we have the following services: Flight booking API, Hotel booking API, and a Customer API.

We'd write the following steps:

from talepy.steps import Step

class DebitCustomerBalance(Step):

    def __init__(self):
        self.payment_client= {}

    def execute(self, state):
        state['payment_id'] = self.payment_client.bill(state.customer_id, state.payment_amount)
        return state
        
    def compensate(self, state):
        self.payment_client.refund(state['payment_id'])
       

and so on for any of the steps needed. Then in whatever is handling the user's request a distributed transaction can be built:

from talepy import run_transaction

run_transaction(
    steps=[
        DebitCustomerBalance(), 
        BookFlight(), 
        BookHotel(), 
        EmailCustomerDetailsOfBooking()
    ],
    starting_state={}
)

If any step along the way fails then the compensate method on each step is called in reverse order until everything is undone.

Steps as Lambdas

For some cases you may not want to create a class for the step. Lambdas can be used directly instead. Extending the previous example:

from talepy import run_transaction

run_transaction(
    steps=[
        DebitCustomerBalance(), 
        BookFlight(),
        lambda _: print("LOG -- The flight has been booked"),
        BookHotel(), 
        EmailCustomerDetailsOfBooking()
    ],
    starting_state={}
)

This new print statement will now execute following a success in BookFlight.

It's also possible to implement compensations by adding another lambda as a tuple pair:

from talepy import run_transaction

run_transaction(
    steps=[
        DebitCustomerBalance(), 
        BookFlight(),
        (
            lambda _: print("LOG -- The flight has been booked"), 
            lambda _: print("LOG -- ┌[ ಠ ▃ ಠ ]┐ something went wrong")
        ),
        BookHotel(), 
        EmailCustomerDetailsOfBooking()
    ],
    starting_state={}
)

Automatic retries

You may also want to try a step a few times before giving up. A convinence function is provided to help out with this. Starting with the initial example. If the hotel booking step is a bit unreliable and we want to try it 3 times:

from talepy import run_transaction
from talepy.retries import attempt_retries

run_transaction(
    steps=[
        DebitCustomerBalance(), 
        BookFlight(),
        attempt_retries(BookHotel(), times=2), 
        EmailCustomerDetailsOfBooking()
    ],
    starting_state={}
)

The book hotel step will now be executed 3 times before the transaction is aborted. Once all these attempts fail the normal compensation logic will be applied.

Async

If you want to make use of async in your steps you will need to import run_transaction from talepy.async_transactions. This can be awaited on and allows async steps and compensations. In addition you can also run_concurrent_transaction where all steps will be executed concurrently. The downside here is that the ordering of the steps isn't guaranteed. This means all steps receive the same starting state.

example

from talepy.async_transactions import run_transaction
from talepy.steps import Step

class AsyncBookFlight(Step):

    async def execute(self, state):
        # do something
        return state
        
    async def compensate(self, state):
        # revert something
        pass
       

await run_transaction(
    step_defs=[
        DebitCustomerBalance(), 
        AsyncBookFlight(),
        EmailCustomerDetailsOfBooking()
    ],
    starting_state={}
)

Concurrent example

from talepy.async_transactions import run_concurrent_transaction
from talepy.steps import Step

class AsyncBookFlight(Step):

    async def execute(self, state):
        # do something
        return state
        
    async def compensate(self, state):
        # revert something
        pass
       

await run_concurrent_transaction(
    steps=[
        DebitCustomerBalance(), 
        AsyncBookFlight(),
        EmailCustomerDetailsOfBooking()
    ],
    starting_state={}
)

Testing / Development

TODO

talepy's People

Contributors

dependabot[bot] avatar meadsteve avatar sourcery-ai[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

talepy's Issues

Persisting saga state

What about instance crash in a middle step? Especially if it was a N of M retry. Who should persist the transaction metadata and how a recovery is supposed to happen?

Add a protocol to StepLike

Python likes duck typing. Using Protocol gives us a neat way of doing this in a safe but flexible fashion.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.