mkDealsBy() is relying on dataclasses to update deep nested field.

there are two better options : python-lenes and pyrsistent

With capability to update values in nested components, it's quite easy to fine tuning the deal structures make best decisions for capital structures.

The call can be described in pure "" and "Formula"

Here: ,{"poolFactor":0.03}

can be rewrited into a Condition : ["poolFactor,">",0.03], which is more expressiveness

To Do:

• refactor on Hastructure level , which the callWhen only accepts a Pre
• to keep backward compatibility, write thin layer of function to convert ,{"poolFactor":0.03} to ["poolFactor,">",0.03]

• Validation on Assumption
• Validation on Waterfall

Most of input to model a deal is just a Python list or map

that would pose certain chanllenage if user has a slight typo.

Using Pydantic will supply a good type hints and documentation support

Discussed in #20

https://absbox-doc.readthedocs.io/en/latest/howto.html#how-to-load-loan-level-data-freddie-mac

pay fee clause is error

need to re-run all the code

Light Trust 20231 Information Memorandum Black.pdf
Light Trust 2023-1 Pricing Term Sheet.pdf
LT2023-1 Investor Report Oct 2023.pdf

The key is just a string list of lens from lenses, it's not best user readable for user.

background

As introduced features in waterfall as well as deal components, we need to build full coverage validation rule to capture potential misuse of the library.

requirement

• capture all potential use error in waterfall section
• add warning/error message when user build model

Generic / SPV is using dataclass, which will render all values from props in the dataclass, which is quite lots of information.

The solution is :

• override the default string representation( __str__) to include only the key information of the deal
• add a new function show()/view() to inspect ALL detail information of the deal

• deal not found error
• Set assumptions
• Run with multi scenario
• Unicode support
• Update docs

https://github.com/yellowbean/Hastructure/blob/e6794b9236f4adc6ae7a30eb5f8ce493a81e055f/src/Expense.hs#L39C16-L39C83

it's not shown in the doc

1. add it to doc
2. add an example

Background

it's easily break if a syntax changed in new release , while the document isn't being updated at timely manner

Propose Solution

add new regression in pytest

Mixed asset won't work

https://absbox-doc.readthedocs.io/en/latest/modeling.html#multiple-pool-with-mixed-assets

Solution

I think the topic should be brought again regarding include all testing codes in to regression of each releases

Using Dot language to annotate the cashflow distribution on the waterfall

Purpose:

• visualization would make user easy to understand the way of cashflow distribution

• serve as double check purpose

• provide downstream usage for issuance documents

• Ref : https://graphviz.org/docs/layouts/dot/

there are lots of input from user.

How to report to user correctly or CLEARLY ?

https://github.com/keleshev/schema

it should provide good enough validator against user input

Context :

• it has dependency on rust.
• numpy/numpy-financial#29, folks at numpy - financial seems not having interset to include xirr

Solution:

Using

• https://github.com/sprksh/finance-calculator
• https://finance-calculator.readthedocs.io/en/latest/usage.html

Use a tree https://rich.readthedocs.io/en/latest/tree.html to present the deal

Background

Some code samples from the document can be easily outdated and trigger error .

To solve this , I would like to include the notebook based testing which will be included in the regression ,then for each new release of absbox, the sample code should work and the output of result should be correct ( or "same" as previous version )

Reference

https://pytest-notebook.readthedocs.io/en/latest/user_guide/tutorial_intro.html

create a function to

• input
  • map with ID ,bond id ,can be CUSIP/ISIN/ China Bond ID
• history
  • True -> include all history snapshots
  • False -> only fetch latest available ( response with warning if it's too old )
• production
  • True -> only fetch deal models with QA
  • False -> any deal models available in the library

update doc

y -> time line axis

x -> a pair of two stacks bars over the projection period
x ->

• asset stack ->
  • performing asset
  • non-performing asset
  • account cash
• liability stack ->
  • tranches balance
  • due interest
  • due fee
  • liquidity support liability

你好！最近在研究ABS和做相关的Tooling。我看了一下你的这个包，感觉很有水平，想请教一下如何上手ABS？有何推荐的书籍课程？感谢大佬 :P

it's not easy to build with feeding all python list/maps into Generic class.

• How about pre-include template deals in the library, then user can pull one of them and tweak the structure ?

make_gnma_deal(....)

• Or , build a smart deal factory, user feeds it with a chain of instructions to describe how deal looks like, then the factory function will build the deal instead.

build_deal(OC=115%,bond=(), waterfall=(....).......

Description
Great work on adding receivables to version 26.5, this is very useful!

Versions:
Hastructure: 0.26.5
Absbox: 0.26.6

from absbox import API
localAPI = API("http://localhost:8081/",lang='english')

receivable = [["Invoice",
              {
                "start":"2024-03-01",
                "originBalance":2000,
                "originAdvance":1500,
                "dueDate":"2024-06-01",
                "feeType":("FixedRate",0.1)
            }
              ,{
                 #"currentBalance":1500,
                # "remainTerm":3.0,
                 "status":"Current"
                 }
                 
                 ]]

x = {'assets':receivable,'cutoffDate':'2024-03-01'}


poolAssump = ("Pool",("Mortgage",None,None,None,None),None,None)
#poolAssump = ("Pool",("Receivable",None,None,None),None,None) # this works
print(localAPI.runPool(x,poolAssump=poolAssump,read=True))     

• Could we get an error if the poolAssump type doesn't match the asset type?
• Does it make sense to have the poolAssump changed to "Invoice" rather than "Receivable" to be consistent.
• Could we clarify in the docs that Receivables only takes in 3 pool assumptions.

Background

By current design, the CDR CPR vector curve is just a python list ,which is not easy way to understand how this vector being applied to pool performance .

Question _0.01 being applied to what asset periods ? when the 0.01 starts ? _

Solution 1: create a util function to create pool performance vector via dates

function buildVector():

buildVector( dealObject, "CDR",  ["<date1>",<val1>],[<date2>,<val2>].....)
# it will drop the data points before projection start date ( cutoff date / and previous collection date ) 
# if the <date1> is after the projection start date , raise error 

conda is fu* up environment which shadows the package management of Python.

This task is being setup to build virtual env for user to avoid dependency issue if user is using 3rd party package management .

The goal is to create separate environment which user can install via pip and the dependency is separated from user's system package library.

if there are two waterfall actions share same paramters and one is in deal status A the other one is in deal status B.

The bug show up

Current

There is only one RuntimeError in the package

The package just throw everything as RuntimeError ,expectially if the error was originated from Hastructure , which is less human friendly to Non-Haskeller .

Propose solution

• Connection Error

• Version Error

• Engine Error

• Invalid Request Message Error for ( Deal / Assumption / Pool / Asset )

• Building type not being matched

  • The "_" clause need to be raised if match fails

Background

it takes couple key stroke to have access to multiple result: deal status change, cashflow , logs, pool flow, pricing
it would be great to have a report generated for user to have an overview of the transaction performance given

Solution

a candy function:

• input from user

  • (MUST) pool performance / deal assumption
  • (Optional) a bond tuple with bond name and position size
  • (Optional) a list of
  • (Optional) output format -> HTML by default , PDF ( if possible )

• process

  • call run deal and read result in pandas dataframe

• output

  • generate a list of DataFrame base on the input
  • optionally render these into HTML/PDF

Recipes

Recipes is a VIEW from a role of following:

• Investor, which has a position of tranches:
• trustee
• originator

pygwalker
pivottablejs
qgrid
itables

it doesn't make much sense for user to type all the strings.

Solution

Create enums for these constants in

• https://absbox-doc.readthedocs.io/en/latest/modeling.html#constants
• https://absbox-doc.readthedocs.io/en/latest/modeling.html#deal-status-enums
• waterfall actions

using string enums to keep backward compatible

Discussed in #5

These already exposed in engine but not yet expose in wrapper

              | RefRate IRate DealStats Float RateReset               -- ^ interest rate depends to a formula
              | CapRate InterestInfo IRate                            -- ^ cap rate 
              | FloorRate InterestInfo IRate                          -- ^ floor rate

https://rich.readthedocs.io/en/latest/tables.html

using static code analyzer

• mypy
• pyright
• pytype

Ref: https://ioflood.com/blog/mypy/#:~:text=Pytype%3A%20Google's%20Static%20Type%20Analyzer&text=In%20this%20example%2C%20Pytype%20is,are%20worthy%20alternatives%20to%20consider.

Goal

setup github action to run static code analyzer on each commit

• https://github.com/jakebailey/pyright-action
• https://github.com/google/pytype
• https://github.com/marketplace/actions/mypy-action

we have prodDealsBy() function to generate a combination of deals. It is quite useful for sensitivity analysis on deal structures

While , it's useful to extend to make it easer in building multiple assumptions.

like build prodAssumpBy()

Hi Xiaoyu
I was trying to test the test01 run from the doc example.

def read_pricing(self, pricing):
if pricing:
return mkComponent(pricing)
return None

mkComponent is not defined in the component.py file

Add a Util function F

Parameter : F( "run result" , { bondName: numberOfPosition} )

Return : { bondName: cashflow }

Awesome work on adding receivables to version 26.5. This is extremely useful!

Versions:
Hastructure: 0.26.5
Absbox: 0.26.6

I've noticed a few issues when trying to add buyAsset of Receivable type into the water fall:
Currently my deal looks like:

receivable1 = ["Invoice"
              ,{"start":"2024-04-01","originBalance":2000
                 ,"originAdvance":1500,"dueDate":"2024-08-01"
                ,"feeType":("Fixed",150)}
              ,{"status":"Current"}]
receivable4 = ["Invoice"
              ,{"start":"2024-04-01","originBalance":2000
                 ,"originAdvance":1500,"dueDate":"2024-06-01"
                ,"feeType":("FactorFee", 0.01, 25, "Up")}
              ,{"status":"Current"}]
pool = {'assets':[receivable1,receivable4],'issuanceStat':{"IssuanceBalance":100000}}

dates={"collect": ["2024-02-29","2024-03-31"]
        ,"pay": ["2024-02-15"  
                      ,"2024-03-15"]  
        ,"stated":"2027-11-04"
        ,"poolFreq":"MonthFirst"
        ,"payFreq":["DayOfMonth",15]
        }
accounts={"collections":{"balance":50_000_000},
            "revolBuyAcc":{"balance":100_000}
            ,"purchaserAccount": {"balance":0}
            ,"sellerAccount": {"balance": 0}
          ,"cashReserve":{"balance":0}}
          
bonds={"A1":{"balance":150_000_000
                    ,"rate":0.15
                    ,"originBalance":200_000_000
                    ,"originRate":0.0858
                    ,"startDate":"2022-11-04"
                    ,"rateType":{"fix": 0.15, "dayCount":"DC_ACT_365A"}
                    ,"maturityDate":"2027-11-04"
                    ,"bondType":{"Lockout":"2023-11-04"}}
                    ,"B":{
                    "balance":1
                    ,"rate":0.0
                    ,"originBalance":1000
                    ,"originRate":0.0
                    ,"startDate":"2022-11-04"
                    ,"rateType":{"Fixed":0.00}
                    ,"bondType":{"Equity":None}
                    }
                    }

fees={"technology_fee":{"type":{"annualPctFee":[("bondBalance",),0.004]}}}

waterfall = {"default": [["payFee","collections",["technology_fee"]]
  ,["accrueAndPayInt","collections",["A1"]]
  ,["transfer" ,"collections","revolBuyAcc"]
  ,["buyAsset",["Current|Defaulted",1.0,0],"revolBuyAcc",None] #This line is the problematic one. If I remove it the server will run the deal
  ,["payIntResidual","collections","B"]]
  
  ,"endOfCollection":[["calcFee","technology_fee"] ,["calcInt","A1"]]

        ,"cleanUp":[["sellAsset", ["Current|Defaulted",1.0,0], "collections"] #I also comment out this line as well
                    ,["transfer" ,"cashReserve",'collections']
                    ,["transfer" ,"revolBuyAcc",'collections']
                    ,["accrueAndPayInt","collections",["A1"]]
                    ,["payPrin","collections",["A1"]]
                    ,["payIntResidual","collections","B"]]}
        
collects=[["CollectedCash","collections"]
              ,["CollectedPrincipal","collections"]
              ,["CollectedFeePaid","collections"]]

trigger=None
deal_data = {
          "name":"Deal"
          ,"dates":dates
          ,"pool":pool
          ,"accounts":accounts
          ,"fees":fees
          ,"bonds":bonds
          ,"waterfall":waterfall
          ,"collect":collects
          ,"trigger":trigger
          ,"status":"Revolving"
        } 

from absbox import mkDeal, API
deal=mkDeal(deal_data)


revol_asset = ["Invoice",
                  {
                    "start":"2021-01-01",
                    "originBalance":20000,
                    "originAdvance":18000,
                    "dueDate":"2021-04-01",
                    "feeType":("FixedRate",0.069)
                }
                  ,{
                    "status":"current"
                    }]
revolving=("revolving"
        ,["constant",revol_asset]
        ,("Pool",("Receivable",None,None,None),None,None))
      
localAPI = API("http://localhost:8081/",lang='english')
r = localAPI.run(deal
              ,poolAssump = ("Pool",("Receivable",None,None,None),None,None)
              ,runAssump = [("inspect"
                            ,("MonthEnd",("status",("Amortizing")))
                            ,("MonthEnd",("accountBalance",))
                            ,("MonthEnd",("poolBalance",))
                            ,("MonthEnd",("bondBalance",))
                            ),revolving,("call",{"afterDate":"2025-11-04"}),
                          ]
            ,read=True)

Could you advise? I'm not entirely sure what the issue is

https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/
https://godatadriven.com/blog/a-practical-guide-to-setuptools-and-pyproject-toml/

• make it backward compatible
• extent to SPV and Generic

it's hardcoded, and require double maintain if there is a new int rate bool type formula were included.

There should be a solution to fix it once for all.

as pyxirr is built by rust and can't be referenced by PyScript

Thanks for the great package first!

Reproducible Example

After import API,
from absbox import API

When I ran below code, I came up with an error.
localAPI = API("https://deal-bench.xyz/api")

Error

---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
Input In [26], in <cell line: 1>()
----> 1 localAPI = API("https://deal-bench.xyz/api")

File <string>:5, in __init__(self, url, version)

File ~\AppData\Local\mambaforge\envs\absbox\lib\site-packages\absbox\client.py:25, in API.__post_init__(self)
     22     return
     24 # print(_r)
---> 25 echo = json.loads(_r)
     26 self.server_info = echo
     27 supported_client_versions = echo['version']

File ~\AppData\Local\mambaforge\envs\absbox\lib\json\__init__.py:346, in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    341     s = s.decode(detect_encoding(s), 'surrogatepass')
    343 if (cls is None and object_hook is None and
    344         parse_int is None and parse_float is None and
    345         parse_constant is None and object_pairs_hook is None and not kw):
--> 346     return _default_decoder.decode(s)
    347 if cls is None:
    348     cls = JSONDecoder

File ~\AppData\Local\mambaforge\envs\absbox\lib\json\decoder.py:337, in JSONDecoder.decode(self, s, _w)
    332 def decode(self, s, _w=WHITESPACE.match):
    333     """Return the Python representation of ``s`` (a ``str`` instance
    334     containing a JSON document).
    335 
    336     """
--> 337     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    338     end = _w(s, end).end()
    339     if end != len(s):

File ~\AppData\Local\mambaforge\envs\absbox\lib\json\decoder.py:355, in JSONDecoder.raw_decode(self, s, idx)
    353     obj, end = self.scan_once(s, idx)
    354 except StopIteration as err:
--> 355     raise JSONDecodeError("Expecting value", s, err.value) from None
    356 return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0) 

Analysis

I tried to figure out the cause by running

import requests
out = requests.get("https://deal-bench.xyz/api",verify=False)
print(out)

I got

<Response [403]>

---> I suppose it was because I can not use the given server "https://deal-bench.xyz/api".
Is there any requirement to use this server? Thanks!

Installed Versions

absbox=0.1.1.5=pypi_0
argon2-cffi=21.3.0=pyhd8ed1ab_0
argon2-cffi-bindings=21.2.0=py310he2412df_2
asttokens=2.0.5=pyhd8ed1ab_0
attrs=22.1.0=pyh71513ae_1
backcall=0.2.0=pyh9f0ad1d_0
backports=1.0=py_2
backports.functools_lru_cache=1.6.4=pyhd8ed1ab_0
beautifulsoup4=4.11.1=pyha770c72_0
bleach=5.0.1=pyhd8ed1ab_0
bzip2=1.0.8=h8ffe710_4
ca-certificates=2022.6.15=h5b45459_0
certifi=2022.6.15=pypi_0
cffi=1.15.1=py310hcbf9ad4_0
charset-normalizer=2.1.0=pypi_0
colorama=0.4.5=pyhd8ed1ab_0
debugpy=1.6.0=py310h8a704f9_0
decorator=5.1.1=pyhd8ed1ab_0
defusedxml=0.7.1=pyhd8ed1ab_0
entrypoints=0.4=pyhd8ed1ab_0
executing=0.10.0=pyhd8ed1ab_0
flit-core=3.7.1=pyhd8ed1ab_0
idna=3.3=pypi_0
importlib-metadata=4.11.4=py310h5588dad_0
importlib_resources=5.9.0=pyhd8ed1ab_0
intel-openmp=2022.1.0=h57928b3_3787
ipykernel=6.15.1=pyh025b116_0
ipython=8.4.0=py310h5588dad_0
ipython_genutils=0.2.0=py_1
jedi=0.18.1=pyhd8ed1ab_2
jinja2=3.1.2=pyhd8ed1ab_1
jsonschema=4.9.1=pyhd8ed1ab_0
jupyter_client=7.3.4=pyhd8ed1ab_0
jupyter_core=4.11.1=py310h5588dad_0
jupyterlab_pygments=0.2.2=pyhd8ed1ab_0
libblas=3.9.0=16_win64_mkl
libcblas=3.9.0=16_win64_mkl
libffi=3.4.2=h8ffe710_5
libiconv=1.16=he774522_0
liblapack=3.9.0=16_win64_mkl
libsodium=1.0.18=h8d14728_1
libsqlite=3.39.2=h8ffe710_1
libxml2=2.9.14=hf5bbc77_4
libxslt=1.1.35=h34f844d_0
libzlib=1.2.12=h8ffe710_2
lxml=4.9.1=py310he2412df_0
markupsafe=2.1.1=py310he2412df_1
matplotlib-inline=0.1.3=pyhd8ed1ab_0
mistune=0.8.4=py310he2412df_1005
mkl=2022.1.0=h6a75c08_874
nbclient=0.6.6=pyhd8ed1ab_0
nbconvert=6.5.3=pyhd8ed1ab_0
nbconvert-core=6.5.3=pyhd8ed1ab_0
nbconvert-pandoc=6.5.3=pyhd8ed1ab_0
nbformat=5.4.0=pyhd8ed1ab_0
nest-asyncio=1.5.5=pyhd8ed1ab_0
notebook=6.4.12=pyha770c72_0
numpy=1.23.1=py310h8a5b91a_0
openssl=3.0.5=h8ffe710_1
orjson=3.7.12=pypi_0
packaging=21.3=pyhd8ed1ab_0
pandas=1.4.3=py310hf5e1058_0
pandoc=2.19=h57928b3_0
pandocfilters=1.5.0=pyhd8ed1ab_0
parso=0.8.3=pyhd8ed1ab_0
pickleshare=0.7.5=py_1003
pip=22.2.2=pyhd8ed1ab_0
pkgutil-resolve-name=1.3.10=pyhd8ed1ab_0
prometheus_client=0.14.1=pyhd8ed1ab_0
prompt-toolkit=3.0.30=pyha770c72_0
psutil=5.9.1=py310he2412df_0
pure_eval=0.2.2=pyhd8ed1ab_0
pycparser=2.21=pyhd8ed1ab_0
pygments=2.12.0=pyhd8ed1ab_0
pyparsing=3.0.9=pyhd8ed1ab_0
pyrsistent=0.18.1=py310he2412df_1
python=3.10.5=hcf16a7b_0_cpython
python-dateutil=2.8.2=pyhd8ed1ab_0
python-fastjsonschema=2.16.1=pyhd8ed1ab_0
python_abi=3.10=2_cp310
pytz=2022.2=pyhd8ed1ab_0
pywin32=303=py310he2412df_0
pywinpty=2.0.7=py310h00ffb61_0
pyzmq=23.2.1=py310h73ada01_0
requests=2.28.1=pypi_0
send2trash=1.8.0=pyhd8ed1ab_0
setuptools=64.0.3=py310h5588dad_0
six=1.16.0=pyh6c4a22f_0
soupsieve=2.3.2.post1=pyhd8ed1ab_0
sqlite=3.39.2=h8ffe710_1
stack_data=0.4.0=pyhd8ed1ab_0
tbb=2021.5.0=h2d74725_1
terminado=0.15.0=py310h5588dad_0
tinycss2=1.1.1=pyhd8ed1ab_0
tk=8.6.12=h8ffe710_0
tornado=6.2=py310he2412df_0
traitlets=5.3.0=pyhd8ed1ab_0
tzdata=2022b=h191b570_0
ucrt=10.0.20348.0=h57928b3_0
urllib3=1.26.11=pypi_0
vc=14.2=hb210afc_6
vs2015_runtime=14.29.30037=h902a5da_6
wcwidth=0.2.5=pyh9f0ad1d_2
webencodings=0.5.1=py_1
wheel=0.37.1=pyhd8ed1ab_0
winpty=0.4.3=4
xz=5.2.6=h8d14728_0
zeromq=4.3.4=h0e60522_1
zipp=3.8.1=pyhd8ed1ab_0