having states and transition why specific transition can't return python object back t

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

Hi <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="

Comments (10)

catchthemonster commented on June 7, 2024 1

Hi Fernando,
thanks for the explanation, will try...
Do we have in docs a kind of quick cheat sheet on what all of the facets of python-statemachine ideas are? Do not know others but for me it is hard to dedicate so much time to read documentation and trying things. I have an extremely hard time figuring out naming conventions for these calls and correlating this info to the problem subject I have. It is easy when you eloquently explained but it would be nice to have one page dedicated to the:
hey you can pass arguments to transition, and action (method) in this way
you can return object( Python is all objects anyway) this and this way
and methods that signify types of sm are on_enter vs. on_ and so forth.

I know I am asking for too much, but the framework and your work is excellent... There is a lot of cool ideas here. But if you want folks to integrate this machinery into their projects you need to explain to them ins and out of the data flow and what can/can not do things for different situations.

many thanks...

from python-statemachine.

iamgodot commented on June 7, 2024 1

@fgmacedo Sorry for necrobumping, but would you like me to refine the return value part of the doc? Or I could do this later since the enhancement is in progress.

from python-statemachine.

fgmacedo commented on June 7, 2024

Hi @catchthemonster , how are you?

The default behavior is to return the value returned by the callbacks to te caller. Please see return values.

If you want more specific help, can you share the code of your state machine definition, so we can inspect if there are any typo or error.

Best!

from python-statemachine.

catchthemonster commented on June 7, 2024

Hi Fernado thank you for being so nice!
I like your framework and I use it in big clustered framework I developed.
That being said here is the code:
driverMain.py
from logging import getLogger
from ProfilingMixIn import AprofileMixIn
from SMachine import SMEngine
from IDriverMain import IdriverMain
logger = getLogger(name)

class PropFailure(Exception):
pass

class Struc:
def init(self, **entries):
self.dict.update(entries)

class MDriver(IdriverMain):
"""
Main application driver responsible for staging state machine workflows to be executed
"""

def __init__(self, profileMe, timeMe):
    self.profileMe = profileMe
    self.timeMe = timeMe
    p = AprofileMixIn(profileMe, timeMe)
    logger.info(AprofileMixIn._classname())
    logger.info(p.__repr__())
    logger.info(p.__getitem__())

def initSM(self, thisApp, thisArgs, logData, env, envSections):
    ## state machine starts here
    domain = SMEngine(state=1)
    workflow = domain.sm
    logger.info("Starting a Warehouse Workflow: current StateMachine state is: {}".format(workflow.current_state))
    workflow.setup(thisApp, thisArgs, logData, env, envSections, profileMe=self.profileMe, timeMe=self.timeMe)
    workflow.caching()
    workflow.vaultCreds()
    workflow.sharedConfigs()
    managedObject = workflow.bootstrap()
    workflow.bootStrapAppManager(managedObject.initAsync, managedObject.runningServices)
    ...

---------------------------------------------

SMachine.py

import ast
import socket
import os
import json
import time
import pickle
import struct
import uuid
import logging
from accessify import private, protected
from statemachine import StateMachine, State
from statemachine.mixins import MachineMixin
from logging import getLogger
from functools import wraps
from types import SimpleNamespace as sns
from CacheInterface import Cinterface as cacheObj
from Dill import DillCache as dillObj
from FailureUtil import DeQueu as deqObj
from memCacheDWrapper import memCacheWrap
from Networking.LSocket import MSocket
from datetime import datetime
from VWrapper import VaultWrapper as vault
from Asynclibs.InitAsyncServices import InitAsyncInit
from ProfilingMixIn import AprofileMixIn
from UDPMemorizedCache import Server
from multiprocessing import Process
from copy import deepcopy
from pydotdict import DotDict
from Manager import AppManager

logger = getLogger(name)

class PropFailure(Exception):
pass

class Struc:
def init(self, **entries):
self.dict.update(entries)

class stateModel(object):
def init(self, state):
self.state = state

def mcClientBootStrap(func):
wraps(func)
def wrapper(self, msObj, sRecv, kvMessage, sd=None, *args, **kwargs):
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
except Exception as mcCErr:
logger.error("Error occurred:{}".format(mcCErr))
with sock:
return func(self, sock, msObj, sRecv, kvMessage, sd=None, *args, **kwargs)
logger.error(f'Failed to get connection.\nconn: {conn}\ndata: {kvMessage}')
return wrapper

class SMEngine(MachineMixin):
state_machine_name = 'SMachine.Transitions'
state_machine_attr = 'sm'
state_field_name = 'workflowStep'
workflowStep = 1

def __init__(self, **kwargs):
    for k, v in kwargs.items():
        setattr(self, k, v)
    super(SMEngine, self).__init__()

def __repr__(self):
    return "{}({!r})".format(type(self).__name__, self.__dict__)

class Transitions(StateMachine):
"""
StateMachine for Warehouse ensures that your system stays in a valid states at all times.
"""

## states
setupConfig = State('setupConfig', initial=True, value=1)
setupCaching = State('setupCaching', value=2)
getVaultSecrets = State('getVaultSecrets', value=3)
setSharedConfigs = State('setSharedConfigs', value=4)
mpBootstrap = State('mpBootstrap', value=5)
bootStrapAppManager = State('bootStrapAppManager', value=6)
## transitions

setup = setupConfig.to.itself()
caching = setupConfig.to(setupCaching)
vaultCreds = setupCaching.to(getVaultSecrets)
sharedConfigs = getVaultSecrets.to(setSharedConfigs)
bootstrap = setSharedConfigs.to(mpBootstrap)
bootAppManager = mpBootstrap.to(bootStrapAppManager)

@protected
@AprofileMixIn.profile
@AprofileMixIn.timeIt
def getVSecrets(self):
    ## setup a switch stament here for better handeling
    if self.authMode == 'ldap':
        self.vaultConf.userTokenURI = self.uCreds.userTokenURI
        ## first get vault token and embed a user token to vault configuration
        self.vaultObj = vault(self.vaultConf, self.authMode)
        if self.vaultObj.userSetup(self.uCreds):
            self.vaultObj.initVault()
        else:
            logger.error("Error occurred: Ldap user setup failed...")
    elif self.authMode.lower() == 'msi':  ## we are going MSI
        self.vaultObj = vault(self.vaultConf, self.authMode, self.profileMe, self.timeMe)
        ## first get vault token MSI
        self.vaultConf.aToken = self.vaultObj.getMSIToken(self.azManagmentApiResource, self.azGeneralHeader,  self.azAuthURL)
        self.aToken = self.vaultConf.aToken
        self.vaultObj.initVault()
    else:
        logger.error("Error occurred: Authentication mode not supplied or incorrectly setup in template...")
        ##todo where we are going from here with proping errors
        ##todo maybe raise can finish gracefuly here with error and exist
        raise PropFailure("Error occurred: Vault secrets unavailable... bailing out...")
    try:
        vaultSecrets = self.vaultObj.getDynamicSecrets(self.vMPoints)
    except Exception as vErr:
        logger.error("Error occurred: {}".format(vErr))
    return self.vSecrets, vaultSecrets

@private
@AprofileMixIn.profile
@AprofileMixIn.timeIt
def normalizePaths(self, *args):
    absPath = None
    if args:
        absPath = self.absPath(list(args))
    else:
        return absPath
    return absPath

@private
@AprofileMixIn.profile
@AprofileMixIn.timeIt
def absPath(self, joinL: list):
    path = None
    try:
        path = os.path.join(*joinL)
    except Exception as osErr:
        logger.error("Error occurred: {}".format(osErr))
    return path

@private
@AprofileMixIn.profile
@AprofileMixIn.timeIt
def toDict(self, obj):
    try:
        a = json.dumps(obj, default=lambda o: o.__dict__)
    except Exception as jErr:
        logger.error(jErr)
    return

@private
@AprofileMixIn.profile
@AprofileMixIn.timeIt
def myDict(self, obj):
    try:
        if not hasattr(obj, "__dict__"):
            return self.toDict(obj)
        result = {}
        for key, val in obj.__dict__.items():
            if key.startswith("_"):
                continue
            element = []
            if isinstance(val, list):
                for item in val:
                    element.append(self.myDict(item))
            else:
                element = self.myDict(val)
            result[key] = element
        return result
    except Exception as mdErr:
        logger.error("Error occurred: {}".format(mdErr))

def mpRun(self, msObj, logData):
    logger = logging.getLogger()
    logger.setLevel(logging.DEBUG)
    logger.info('Starting service for %s', msObj.service)
    logger.info("Starting Key:Value memorization Server...")
    Server(msObj.service, msObj.host, int(msObj.port))
    logger.info("Starting HashTableService")


@AprofileMixIn.profile
@AprofileMixIn.timeIt
def memorizeCacheBootStrap(self, msObj, logData):
    try:
        proc = Process(target=self.mpRun, args=(msObj, logData), daemon=True)
        proc.start()
        self.memCBSproc[proc.pid] = proc
    except Exception as mcbsErr:
        logger.error("Error occurred: {}".format(mcbsErr))
        raise (PropFailure(mcbsErr))
    return msObj.service, self.memCBSproc

@mcClientBootStrap
def mcClinet(self, sock, mcObj, sRecv, kvMessage, sd=None):
    if not sd:
        data = None
        try:
            data = pickle.dumps(kvMessage)
            # As you can see, there is no connect() call; UDP has no connections.
            # Instead, data is directly sent to the recipient via sendto().
            sock.sendto(data, (mcObj.host, int(mcObj.port)))
            received = sock.recv(int(sRecv))
            rdata = pickle.loads(received)
            logger.info("Sent:     {}".format(data))
            logger.info("Received: {}".format(rdata))
        except Exception as mcCErr:
            logger.error("Error occurred:{}".format(mcCErr))
        return data
    elif sd == 'shutdown':
        try:
            data = pickle.dumps(sd)
            sock.sendto(data, (mcObj.mcServer, mcObj.mcPort))
        except Exception as mcCErr:
            logger.error("Error occurred:{}".format(mcCErr))

@AprofileMixIn.profile
@AprofileMixIn.timeIt
def createMCObj(self, service, action, value, serializer):
    cmo = sns()
    cmo.key = str(uuid.uuid4())
    cmo.service = service
    cmo.action = action
    cmo.ts = f'{datetime.now():%Y-%m-%d>%H:%M:%S%z}'
    cmo.value = value
    cmo.serializer = serializer
    return cmo

@protected
@AprofileMixIn.profile
@AprofileMixIn.timeIt
def anySettigns(self, services, templates, logData):
    ## wrap all the necessery data for service transport
    eData = sns()
    eData.logData = logData
    globalSettings = dict()
    for app in services:
        eData.template = getattr(templates, app.service)
        ##check that we need to do a deepcopy here
        eData.mySettings = self.myDict(self.sharedConfig)
        globalSettings[app.service] = deepcopy(eData)
    return globalSettings

@protected
@AprofileMixIn.profile
@AprofileMixIn.timeIt
def sourceTemplates(self, conf, templates):
    sources = list()
    allIOSources = ast.literal_eval(conf.__dict__.pop('allIOSources'))
    for service, serviceConf in allIOSources.items():
        logger.info("Setting up service {}".format(service))
        if hasattr(conf, serviceConf):
            try:
                vService = getattr(conf, serviceConf)
                vService = DotDict(ast.literal_eval(vService))
                vService.template = getattr(templates, serviceConf)
            except Exception as objErr:
                logger.error("Error occurred: {}".format(objErr))
            sources.append(vService)
        else:
            logger.info("Service {} config not found.\nService {} will not be started".format(serviceConf, service))
    return sources

@AprofileMixIn.profile
@AprofileMixIn.timeIt
def bootStrap(self, conf, templates, logData):
    """
    Instantiate any decoupled service
    Get templates for the service and converte them to PY objects
    Initialize service MP startup as Daemon
    Collect PID stats and return
    """
    anyService = None
    anySettings = None
    try:
        anyService = self.sourceTemplates(conf, templates)
    except Exception as anyServiceErr:
        logger.error("Error occurred:{}".format(anyServiceErr))
    try:
        anySettings = self.anySettigns(anyService, templates, logData)
    except Exception as anySettingsErr:
        logger.error("Error occurred:{}".format(anySettingsErr))
    try:
        initAsync = InitAsyncInit(self.profileMe, self.timeMe, anyService, anySettings, conf)
        runningServices = initAsync.start()
        logger.info('Delaying to allow service to start...')
        time.sleep(int(self.appManager.delayBootStrap))
    except Exception as initAsyncErr:
        logger.error('{}:{}'.format(initAsyncErr, initAsyncErr.__cause__))
        return None, None
    return initAsync, runningServices

def on_enter_setupConfig(self, thisApp, thisArgs, logData, env, envSections, profileMe, timeMe):
    self.profileMe = profileMe
    self.timeMe = timeMe
    AprofileMixIn(self.profileMe, self.timeMe)
    self.listMemo = memo = self.createMCObj(service='', action='list', value='', serializer='')
    self.thisApp = thisApp
    self.thisArgs = thisArgs
    self.appManager = None
    self.sternSyncManager = dict()
    self.memCached = None
    self.sharedConfig = sns()
    self.mSocket = MSocket()
    self.logData = logData
    ## a full WH conf objects
    self.appManager = env.APP
    ## memorization harness for serialized key:value pairs
    self.memCache = env.MEMORIZATION
    ## Global Paths Purge Paths
    self.rPaths = Struc(**ast.literal_eval(self.thisApp.PATHS.rPaths))
    self.purgePaths = ast.literal_eval(self.thisApp.PATHS.purgePaths)
    ##transport protocols
    self.tProtocol = env.TPROTOCOL
    self.tPTMPLATES = env.TPTMPLATES
    ## valut
    self.authMode = env.APP.authMode
    self.vaultConf = env.VAULT
    self.vMPoints = env.VAULTMOUNTPOINTS
    self.vaultSecrets = None
    self.vaultObj = None
    self.vaultKey = None
    self.vaultSecrets = None
    self.uCreds = env.CREDS
    self.azManagmentApiResource = ast.literal_eval(self.vaultConf.azManagmentApiResource)
    self.azGeneralHeader = self.vaultConf.azGeneralHeader
    self.azAuthURL = self.vaultConf.azAuthURL
    self.vSecrets = self.vaultConf.vaultSecrets
    ## Async object configurations (no WEB)
    self.inputAsync = env.INPUTASYNC
    self.inputAsyncServices = env.INPUTSERVICES
    ## backend WH REST API WEB SERVICE
    self.restApiAsync = env.RESTAPIASYNC
    self.restApiAsyncServices = env.RESTAPISERVICES
    ## sync manager command interface
    self.syncManager = env.SYNCMANAGER
    ##backend Warehouse auth PASETO
    self.pasetoConf = env.PASETO
    ## LDAP configurations
    self.ldapConf = env.LDAP
    ##statistics
    self.statistics = type('statisticsConfig', (object,), {})()
    ## all caching infrastructure
    self.aDill = env.DILLCACHE
    self.aDisk = env.DISKCACHE
    self.aFailure = env.FAILURECACHE
    self.aMemCache = env.MEMCACHED
    ##process PID container for memorization Cache
    self.memCBSproc = dict()

@AprofileMixIn.profile
@AprofileMixIn.timeIt
def on_enter_setupCaching(self):
    ##init dill caches object
    self.dillPathRoot = self.normalizePaths(self.rPaths.BASEPATH, self.aDill.dPath)
    self.dillEngine = dillObj(self.profileMe, self.timeMe, self.dillPathRoot)
    logger.info("dillEngine object name: {}".format(self.dillEngine.__class__))
    ##init disk based caches
    dCachePathRoot = self.normalizePaths(self.rPaths.BASEPATH, self.aDisk.path)
    self.aDisk.path = dCachePathRoot
    self.diskEngine = cacheObj(self.profileMe, self.timeMe, self.aDisk)
    logger.info("diskEngine object name: {}".format(self.diskEngine.__class__))
    ##init failure caches
    self.aFailurePathRoot = self.normalizePaths(self.rPaths.BASEPATH, self.aFailure.path)
    self.failEngine = deqObj(self.profileMe, self.timeMe, self.aFailurePathRoot)
    logger.debug("failEngine object name: {}".format(self.failEngine.__class__))
    ##init stats engine
    self.statistics.cache = self.diskEngine
    ##init memcached
    self.memCached = memCacheWrap(self.aMemCache.__dict__)
    self.memCached.config()
    self.mcdClient = self.memCached.start()
    ## Bellow clinet-server (socket based memorization hashtable)
    ## is bootstrapped as a separate Python process
    ## it is not managed by Flask service manager
    ## todo Setup a monitor for this service independently in code base (to be determined where?)
    ## start memCache for key:value memorization
    memoService, self.memCBSproc = self.memorizeCacheBootStrap(self.memCache, self.logData)
    time.sleep(5)

@AprofileMixIn.profile
@AprofileMixIn.timeIt
def on_enter_getVaultSecrets(self):
    ##get all vault secrets
    self.vaultKey, self.vaultSecrets = self.getVSecrets()
    # serialize secrets
    self.mcdClient.set(self.vaultKey, self.vaultSecrets)
    memo = self.createMCObj(service='vault', action='set', value=self.vaultKey, serializer='memCached')
    ## memorize
    res = self.mcClinet(self.memCache, self.memCache.bRange, memo)
    logger.info('All Application secret cached successfully, memo serialized: {}'.format(pickle.loads(res)))
    logger.info('Memo List: {}'.format(self.mcClinet(self.memCache, self.memCache.bRange, self.listMemo)))

@AprofileMixIn.profile
@AprofileMixIn.timeIt
def on_enter_setSharedConfigs(self):
    ##shared configurations across all assets in the WH are build bellow
    ## Idea here is to not re-bootstrap memory and disk and other caches
    ## vault secrets and anything else that is one time opearation
    ## however not every service configuration needs everything - it depends on what services needs what?
    ## this is heavily encrypted payload for transfer across the wire using varaety of protocols
    ## eg. pyro5, SyncManager api, restAPI, rpcy and others, and it is completely driven by
    ### developer and ini configurations - templates
    ## add py objects to shared config
    self.sharedConfig.rPaths = self.rPaths
    self.sharedConfig.dillPathRoot = self.dillPathRoot
    self.sharedConfig.aDisk = self.aDisk
    self.sharedConfig.aFailurePathRoot = self.aFailurePathRoot
    self.sharedConfig.aMemCache = self.aMemCache
    self.sharedConfig.vault = self.vaultSecrets
    self.sharedConfig.ldap = self.ldapConf
    self.sharedConfig.paseto = self.pasetoConf

    ##below is if we want REST api to sent encrypted payload of py object to service upon initialization
    # self.asynIOConf = sns()
    # self.asynIOConf.protocolSchema = self.restApiAsync.protocolSchema
    # self.asynIOConf.salt = self.restApiAsync.salt
    # self.asynIOConf.postSettings = self.restApiAsync.postSettings
    # delattr(self.restApiAsync, 'protocolSchema')
    # delattr(self.restApiAsync, 'salt')
    # delattr(self.restApiAsync, 'postSettings')
    ## Bootstrap dynamically anything you want asynchronosly to run
    ## you can define which message busses you want to use and what harness you want to bootstrap
    ##init payload and messages busses that move payload and transfer messages between Dispersed Computing (DC) resources

    transport = sns()
    transport.tProtocol = self.tProtocol
    transport.tPTMPLATES = self.tPTMPLATES

##@AprofileMixIn.profile
##@AprofileMixIn.timeIt
def on_enter_mpBootstrap(self):
    ## sync managedObj
    managedObj = sns()
    managedObj.initAsync, managedObj.runningServices = self.bootStrap(self.tProtocol, self.tPTMPLATES, self.logData)
    return managedObj

@AprofileMixIn.profile
@AprofileMixIn.timeIt
def on_enter_bootStrapAppManager(self, asyncServices, runningServices):
    ## bootstraping can only happen for Flask and sync services at the ed of everything being done
    ## we also need to have a registry of whatever is started that can be injected
    ## to flask postmortem of bootstraping ...
    ## for now just keep it to keep app up and running
    ## setup a main application synchronous Manager sync service
    try:
        if self.syncManager.host == '0.0.0.0':
            ##get remote IP to bind services to that IP
            hName, hIP4 = self.mSocket.whereIAm()
            if hName == None or hIP4 == None:
                logger.warning("Warning occurred: Both hostname and hostIP4 unavailable")
            else:
                if ast.literal_eval(self.syncManager.runPublic):
                    self.syncManager.host = hIP4

        self.syncManager.port = int(self.syncManager.port)
        self.syncManager.debug = ast.literal_eval(self.syncManager.debug)
        self.syncManager.runit = ast.literal_eval(self.syncManager.runIt)
        ##here inject memcached configuration
        self.syncManager.memCached = self.memCached

    except Exception as syncManagerErr:
        logger.error("Error occurred: {}".format(syncManagerErr))
    if self.syncManager.runit:
        try:
            self.syncManager.services = asyncServices
            self.syncManager.runningServices = runningServices
            appManager = AppManager(self.profileMe, self.timeMe, self.syncManager.__dict__)
            appManager.start()
        except Exception as appManagerErr:
            logger.error("Error occurred: {}".format(appManagerErr))

Debug
session in PyCharm:

Log:
Connected to pydev debugger (build 231.9011.38)

Debugging enabled!

Template folder: someAppConf

Namespace(environment='someAppConf', d=True)

Application start time in seconds since the epoch as a floating point number 1688044472.7724187

No restriction configuration for /blah

For project someAppConf found template /blah/apps/someApp/ptemplates/someAppConf.conf

Creating directory structure for /blah/logs/someApp

Initializing logging

/python3.10/logging/config.py:183 disabled the PIL.Image logger

/python3.10/logging/config.py:183 disabled the PIL.PngImagePlugin logger

/python3.10/logging/config.py:183 disabled the ProfilingMixIn logger

/python3.10/logging/config.py:183 disabled the asyncio logger

/python3.10/logging/config.py:183 disabled the concurrent.futures logger

/python3.10/logging/config.py:183 disabled the matplotlib logger

/python3.10/logging/config.py:183 disabled the matplotlib.ticker logger

/python3.10/logging/config.py:183 disabled the pkg_resources.extern.packaging.tags logger

2023-06-29 09:14:32,785 - INFO - 2615119 - NMyLogging.py:33 NMyLogging:setup - Logging Initialized

2023-06-29 09:14:32,785 - DEBUG - 2615119 - NMyLogging.py:34 NMyLogging:setup - Debugging is on

2023-06-29 09:14:32,786 - INFO - 2615119 - LightUtil.py:83 LightUtil:postLogInit - Logger recreated.

2023-06-29 09:14:33,862 - INFO - 2615119 - driverMain.py:33 driverMain:init - AprofileMixIn

2023-06-29 09:14:33,863 - INFO - 2615119 - driverMain.py:34 driverMain:init - repr.AprofileMixIn:profileMe=False:timeMe=True

2023-06-29 09:14:33,863 - INFO - 2615119 - ProfilingMixIn.py:43 ProfilingMixIn:getitem - Methods: getitem,Aprofile:{'getitem', 'init', 'repr'}

2023-06-29 09:14:33,863 - INFO - 2615119 - driverMain.py:35 driverMain:init - AprofileMixIn

2023-06-29 09:14:45,714 - INFO - 2615119 - driverMain.py:43 driverMain:initSM - Starting a Warehouse Workflow: current StateMachine state is: State('setupConfig', id='setupConfig', value=1, initial=True, final=False)

2023-06-29 09:14:45,716 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - createMCObj method executed in 0.00 sec

2023-06-29 09:14:45,718 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - absPath method executed in 0.00 sec

2023-06-29 09:14:45,718 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - normalizePaths method executed in 0.00 sec

2023-06-29 09:14:45,718 - INFO - 2615119 - SMachine.py:417 SMachine:on_enter_setupCaching - dillEngine object name: <class 'Dill.DillCache'>

2023-06-29 09:14:45,719 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - absPath method executed in 0.00 sec

2023-06-29 09:14:45,719 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - normalizePaths method executed in 0.00 sec

2023-06-29 09:14:45,719 - INFO - 2615119 - Nio.py:132 Nio:ifExists - Absolute path found... /blah/dCaches

2023-06-29 09:14:45,746 - INFO - 2615119 - DCache.py:43 DCache:init - Successfully initiated disk(LightSQL) cache: /blah/dCaches

2023-06-29 09:14:45,746 - INFO - 2615119 - SMachine.py:423 SMachine:on_enter_setupCaching - diskEngine object name: <class 'CacheInterface.Cinterface'>

2023-06-29 09:14:45,746 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - absPath method executed in 0.00 sec

2023-06-29 09:14:45,746 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - normalizePaths method executed in 0.00 sec

2023-06-29 09:14:45,774 - INFO - 2615119 - FailureUtil.py:33 FailureUtil:init - Successfully initiated double-ended queue: /blah/fCaches

2023-06-29 09:14:45,774 - DEBUG - 2615119 - SMachine.py:428 SMachine:on_enter_setupCaching - failEngine object name: <class 'FailureUtil.DeQueu'>

2023-06-29 09:14:45,774 - INFO - 2615119 - memCacheDWrapper.py:49 memCacheDWrapper:init - Memcache wrapper successfully initialized

2023-06-29 09:14:45,775 - INFO - 2615119 - memCacheDWrapper.py:90 memCacheDWrapper:start - Memcache host:port is "localhost:11211"

2023-06-29 09:14:45,775 - INFO - 2615119 - memCacheDWrapper.py:93 memCacheDWrapper:start - Initiating a regular memcached client...

2023-06-29 09:14:45,783 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - memorizeCacheBootStrap method executed in 0.01 sec

2023-06-29 09:14:45,992 - INFO - 2615326 - SMachine.py:207 SMachine:mpRun - Starting service for memSerialCache

2023-06-29 09:14:45,993 - INFO - 2615326 - SMachine.py:208 SMachine:mpRun - Starting Key:Value memorization Server...

2023-06-29 09:14:45,993 - DEBUG - 2615326 - UDPMemorizedCache.py:119 UDPMemorizedCache:init - Starting : memSerialCache

2023-06-29 09:14:50,789 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - on_enter_setupCaching method executed in 5.07 sec

2023-06-29 09:14:50,792 - DEBUG - 2615119 - VWrapper.py:52 VWrapper:getMSIToken - cpStern:getMSIToken

2023-06-29 09:14:50,793 - DEBUG - 2615119 - AZAuth.py:31 AZAuth:getAnyToken - MSI connection string is: http://...metadata/identity/oauth2/token ...
2023-06-29 09:14:50,807 - DEBUG - 2615119 - connectionpool.py:244 connectionpool:_new_conn - Starting new HTTP connection (1):...

2023-06-29 09:14:50,818 - DEBUG - 2615119 - connectionpool.py:546 connectionpool:_make_request - http://...

2023-06-29 09:14:50,820 - DEBUG - 2615119 - AZAuth.py:47 AZAuth:getAnyToken - My status code is: 200

2023-06-29 09:14:50,820 - DEBUG - 2615119 - AZAuth.py:48 AZAuth:getAnyToken - My return header is: application/json; charset=utf-8

2023-06-29 09:14:50,820 - DEBUG - 2615119 - AZAuth.py:51 AZAuth:getAnyToken - Full content as JSON is: b'{"access_token":"1234....

2023-06-29 09:14:50,821 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - getMSIToken method executed in 0.03 sec

2023-06-29 09:14:50,826 - DEBUG - 2615119 - connectionpool.py:1048 connectionpool:_new_conn - Starting new HTTPS connection (1): vault...:443

2023-06-29 09:14:50,902 - DEBUG - 2615119 - connectionpool.py:546 connectionpool:_make_request - https://vault-e.someAppConf.azure.td.com:443 "POST /../login HTTP/1.1" 200 691

2023-06-29 09:14:50,908 - DEBUG - 2615119 - connectionpool.py:546 connectionpool:_make_request - https://vault-e.someAppConf.azure.td.com:443 "GET /. HTTP/1.1" 200 827

2023-06-29 09:14:50,908 - DEBUG - 2615119 - AZHvaultClient.py:93 AZHvaultClient:vLogin - Auth to vault passed: True

2023-06-29 09:14:50,909 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - initVault method executed in 0.09 sec

2023-06-29 09:14:50,914 - DEBUG - 2615119 - connectionpool.py:546 connectionpool:_make_request - https://vault-e.someAppConf.azure.td.com:443 "GET ... HTTP/1.1" 200 321

2023-06-29 09:14:50,920 - DEBUG - 2615119 - connectionpool.py:546 connectionpool:_make_request - https://vault-e.someAppConf.azure.td.com:443 "GET ...HTTP/1.1" 200 479

2023-06-29 09:14:50,925 - DEBUG - 2615119 - connectionpool.py:546 connectionpool:_make_request - https://vault-e.someAppConf.azure.td.com:443 "GET .... HTTP/1.1" 200 354

2023-06-29 09:14:50,931 - DEBUG - 2615119 - connectionpool.py:546 connectionpool:_make_request - https://vault-e.someAppConf.azure.td.com:443 "GET ... HTTP/1.1" 200 326

2023-06-29 09:14:50,936 - DEBUG - 2615119 - connectionpool.py:546 connectionpool:_make_request - https://vault-e.someAppConf.azure.td.com:443 "GET ...200 319

2023-06-29 09:14:50,942 - DEBUG - 2615119 - connectionpool.py:546 connectionpool:_make_request - https://vault-e.someAppConf.azure.td.com:443 "GET ... 200 326

2023-06-29 09:14:50,947 - DEBUG - 2615119 - connectionpool.py:546 connectionpool:_make_request - https://vault-e.someAppConf.azure.td.com:443 "GET ...200 320

2023-06-29 09:14:50,952 - DEBUG - 2615119 - connectionpool.py:546 connectionpool:_make_request - https://vault-e.someAppConf.azure.td.com:443 "GET ... 200 380

2023-06-29 09:14:50,953 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - getDynamicSecrets method executed in 0.04 sec

2023-06-29 09:14:50,954 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - getVSecrets method executed in 0.16 sec

2023-06-29 09:14:50,955 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - createMCObj method executed in 0.00 sec

2023-06-29 09:14:50,955 - DEBUG - 2615326 - UDPMemorizedCache.py:98 UDPMemorizedCache:handle - Connection '127.0.0.1'

2023-06-29 09:14:50,956 - DEBUG - 2615326 - UDPMemorizedCache.py:101 UDPMemorizedCache:handle - Received data: namespace(key='7c5ca754-848f-4fe4-a435-e0ef264bd248', service='...', action='set', ts='2023-06-29>09:14:50', value='...', serializer='memCached')

2023-06-29 09:14:50,957 - INFO - 2615326 - UDPMemorizedCache.py:65 UDPMemorizedCache:commands - set action called

2023-06-29 09:14:50,957 - DEBUG - 2615326 - UDPMemorizedCache.py:109 UDPMemorizedCache:handle - Sent response:0

2023-06-29 09:14:50,957 - INFO - 2615119 - SMachine.py:241 SMachine:mcClinet - Sent: b'\x80\x04\x95\xbe\x00\x00\x00\x00\x00\x00\x00\x8c\x05types\x94\x8c\x0fSimpleNamespace\x94\x93\x94)R\x94}\x94(\x8c\x03key\x94\x8c$7c5ca754-848f-4fe4-a435-e0ef264bd248\x94\x8c\x07service\x94\x8c\x05vault\x94\x8c\x06action\x94\x8c\x03set\x94\x8c\x02ts\x94\x8c\x132023-06-29>09:14:50\x94\x8c\x05value\x94\x8c\x0cvaultSecrets\x94\x8c\nserializer\x94\x8c\tmemCached\x94ub.'

2023-06-29 09:14:50,958 - DEBUG - 2615326 - UDPMemorizedCache.py:113 UDPMemorizedCache:handle - Closing socket

2023-06-29 09:14:50,958 - INFO - 2615119 - SMachine.py:242 SMachine:mcClinet - Received: 0

2023-06-29 09:14:50,958 - INFO - 2615119 - SMachine.py:458 SMachine:on_enter_getVaultSecrets - All Application secret cached successfully, memo serialized: namespace(key='7c5ca754-848f-4fe4-a435-e0ef264bd248', service='...', action='set', ts='2023-06-29>09:14:50', value='...', serializer='memCached')

2023-06-29 09:14:50,958 - DEBUG - 2615326 - UDPMemorizedCache.py:98 UDPMemorizedCache:handle - Connection '127.0.0.1'

2023-06-29 09:14:50,959 - DEBUG - 2615326 - UDPMemorizedCache.py:101 UDPMemorizedCache:handle - Received data: namespace(key='31773bae-38ab-4cbd-ab24-1475c7f07df2', service='', action='list', ts='2023-06-29>09:14:45', value='', serializer='')

2023-06-29 09:14:50,959 - INFO - 2615326 - UDPMemorizedCache.py:72 UDPMemorizedCache:commands - list action called

2023-06-29 09:14:50,959 - DEBUG - 2615326 - UDPMemorizedCache.py:109 UDPMemorizedCache:handle - Sent response:[('...', '7c5ca754-848f-4fe4-a435-e0ef264bd248')]

2023-06-29 09:14:50,959 - INFO - 2615119 - SMachine.py:241 SMachine:mcClinet - Sent: b'\x80\x04\x95\xa3\x00\x00\x00\x00\x00\x00\x00\x8c\x05types\x94\x8c\x0fSimpleNamespace\x94\x93\x94)R\x94}\x94(\x8c\x03key\x94\x8c$31773bae-38ab-4cbd-ab24-1475c7f07df2\x94\x8c\x07service\x94\x8c\x00\x94\x8c\x06action\x94\x8c\x04list\x94\x8c\x02ts\x94\x8c\x132023-06-29>09:14:45\x94\x8c\x05value\x94h\x08\x8c\nserializer\x94h\x08ub.'

2023-06-29 09:14:50,959 - DEBUG - 2615326 - UDPMemorizedCache.py:113 UDPMemorizedCache:handle - Closing socket

2023-06-29 09:14:50,959 - INFO - 2615119 - SMachine.py:242 SMachine:mcClinet - Received: [('...', '7c5ca754-848f-4fe4-a435-e0ef264bd248')]

2023-06-29 09:14:50,959 - INFO - 2615119 - SMachine.py:459 SMachine:on_enter_getVaultSecrets - Memo List: b'\x80\x04\x95\xa3\x00\x00\x00\x00\x00\x00\x00\x8c\x05types\x94\x8c\x0fSimpleNamespace\x94\x93\x94)R\x94}\x94(\x8c\x03key\x94\x8c$31773bae-38ab-4cbd-ab24-1475c7f07df2\x94\x8c\x07service\x94\x8c\x00\x94\x8c\x06action\x94\x8c\x04list\x94\x8c\x02ts\x94\x8c\x132023-06-29>09:14:45\x94\x8c\x05value\x94h\x08\x8c\nserializer\x94h\x08ub.'

2023-06-29 09:14:50,960 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - on_enter_getVaultSecrets method executed in 0.17 sec

2023-06-29 09:14:50,960 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - on_enter_setSharedConfigs method executed in 0.00 sec

2023-06-29 09:14:50,961 - INFO - 2615119 - SMachine.py:291 SMachine:sourceTemplates - Setting up service TPyro

2023-06-29 09:14:50,961 - INFO - 2615119 - SMachine.py:291 SMachine:sourceTemplates - Setting up service TRest

2023-06-29 09:14:50,961 - INFO - 2615119 - SMachine.py:291 SMachine:sourceTemplates - Setting up service TRPC

2023-06-29 09:14:50,962 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - sourceTemplates method executed in 0.00 sec

2023-06-29 09:14:51,027 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - anySettigns method executed in 0.07 sec

2023-06-29 09:14:53,426 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - encrypt method executed in 2.39 sec

2023-06-29 09:14:53,427 - INFO - 2615119 - InitAsyncServices.py:28 InitAsyncServices:start - Loading settings for tPyro

2023-06-29 09:14:53,428 - INFO - 2615119 - InitAsyncServices.py:31 InitAsyncServices:start - Invoking asynFactory for tPyro

2023-06-29 09:14:53,459 - INFO - 2615119 - InitAsyncServices.py:28 InitAsyncServices:start - Loading settings for tRest

2023-06-29 09:14:53,460 - INFO - 2615119 - InitAsyncServices.py:31 InitAsyncServices:start - Invoking asynFactory for tRest

2023-06-29 09:14:53,492 - INFO - 2615119 - InitAsyncServices.py:28 InitAsyncServices:start - Loading settings for tRPC

2023-06-29 09:14:53,493 - INFO - 2615119 - InitAsyncServices.py:31 InitAsyncServices:start - Invoking asynFactory for tRPC

2023-06-29 09:14:53,525 - INFO - 2615119 - ProfilingMixIn.py:75 ProfilingMixIn:impl - start method executed in 0.10 sec

2023-06-29 09:14:53,525 - INFO - 2615119 - SMachine.py:328 SMachine:bootStrap - Delaying to allow service to start...

Creating directory structure for /blah/logs/someApp

Initializing logging

/python3.10/logging/config.py:183 disabled the AZAuth logger

/python3.10/logging/config.py:183 disabled the AZAuthcore logger

/python3.10/logging/config.py:183 disabled the AZHvaultClient logger

/python3.10/logging/config.py:183 disabled the AZSearchFilters logger

/python3.10/logging/config.py:183 disabled the Args.LightUtil logger

/python3.10/logging/config.py:183 disabled the AsyncBootStrap logger

/python3.10/logging/config.py:183 disabled the Asynclibs.AsyncBootStrap logger

/python3.10/logging/config.py:183 disabled the Asynclibs.InitAsyncServices logger

/python3.10/logging/config.py:183 disabled the CacheInterface logger

/python3.10/logging/config.py:183 disabled the CoreManagerRestMixIn logger

/python3.10/logging/config.py:183 disabled the DCache logger

/python3.10/logging/config.py:183 disabled the Dill logger

/python3.10/logging/config.py:183 disabled the Encrypt.MessageHash logger

/python3.10/logging/config.py:183 disabled the EndPoints logger

/python3.10/logging/config.py:183 disabled the FailureUtil logger

/python3.10/logging/config.py:183 disabled the Io.Nio logger

/python3.10/logging/config.py:183 disabled the Manager logger

/python3.10/logging/config.py:183 disabled the ManagerRest logger

/python3.10/logging/config.py:183 disabled the Mixins.AsyncInitMixIn logger

/python3.10/logging/config.py:183 disabled the Networking.LSocket logger

/python3.10/logging/config.py:183 disabled the Nio logger

/python3.10/logging/config.py:183 disabled the PIL.Image logger

/python3.10/logging/config.py:183 disabled the PIL.PngImagePlugin logger

/python3.10/logging/config.py:183 disabled the ProfilingMixIn logger

/python3.10/logging/config.py:183 disabled the ProtocolTransport logger

/python3.10/logging/config.py:183 disabled the SMachine logger

/python3.10/logging/config.py:183 disabled the Server logger

/python3.10/logging/config.py:183 disabled the VWrapper logger

/python3.10/logging/config.py:183 disabled the asyncio logger

/python3.10/logging/config.py:183 disabled the charset_normalizer logger

/python3.10/logging/config.py:183 disabled the concurrent.futures logger

/python3.10/logging/config.py:183 disabled the dill logger

/python3.10/logging/config.py:183 disabled the driverMain logger

/python3.10/logging/config.py:183 disabled the hvac.api.auth_methods.aws logger

/python3.10/logging/config.py:183 disabled the hvac.api.auth_methods.azure logger

/python3.10/logging/config.py:183 disabled the hvac.api.auth_methods.gcp logger

/python3.10/logging/config.py:183 disabled the hvac.api.secrets_engines.identity logger

/python3.10/logging/config.py:183 disabled the hvac.api.secrets_engines.kv logger

/python3.10/logging/config.py:183 disabled the hvac.api.system_backend logger

/python3.10/logging/config.py:183 disabled the hvac.api.system_backend.system_backend_mixin logger

/python3.10/logging/config.py:183 disabled the hvac.api.vault_api_base logger

/python3.10/logging/config.py:183 disabled the hvac.api.vault_api_category logger

/python3.10/logging/config.py:183 disabled the matplotlib logger

/python3.10/logging/config.py:183 disabled the matplotlib.ticker logger

/python3.10/logging/config.py:183 disabled the memCacheDWrapper logger

/python3.10/logging/config.py:183 disabled the memCacheInterface logger

/python3.10/logging/config.py:183 disabled the pkg_resources.extern.packaging.tags logger

/python3.10/logging/config.py:183 disabled the pymemcache.client.hash logger

/python3.10/logging/config.py:183 disabled the requests logger

/python3.10/logging/config.py:183 disabled the socks logger

/python3.10/logging/config.py:183 disabled the urllib3 logger

/python3.10/logging/config.py:183 disabled the urllib3.connection logger

/python3.10/logging/config.py:183 disabled the urllib3.connectionpool logger

/python3.10/logging/config.py:183 disabled the urllib3.poolmanager logger

/python3.10/logging/config.py:183 disabled the urllib3.response logger

/python3.10/logging/config.py:183 disabled the urllib3.util.retry logger

/python3.10/logging/config.py:183 disabled the uvicorn.error logger

/python3.10/logging/config.py:183 disabled the waitress logger

/python3.10/logging/config.py:183 disabled the waitress.queue logger

/python3.10/logging/config.py:183 disabled the watchfiles.main logger

/python3.10/logging/config.py:183 disabled the watchfiles.watcher logger

2023-06-29 09:14:53,572 - INFO - 2615387 - NMyLogging.py:33 NMyLogging:setup - Logging Initialized

2023-06-29 09:14:53,573 - DEBUG - 2615387 - NMyLogging.py:34 NMyLogging:setup - Debugging is on

2023-06-29 09:14:53,573 - INFO - 2615387 - AsyncBootStrap.py:33 AsyncBootStrap:run - Starting service for tPyroAPI