tukl-msd / finance.benchmark Goto Github PK

From the document:

Portability: Related to the previous aspect. How difficult is it to re-deploy an existing solution onto a different device of the same architecture / a new device of a next generation architecture / a completely different architecture?

The time it takes to design or redesign a solver is a significant consideration as developers need to get working results quickly. Many applications are also frequently tweaked and modified.

• Flexibility
  • give penalty points for products that cannot be priced (formal)?
  • only text description?

From the document:

Feasibility

In order for this to meaningful, that means that we need to have reference prices for all tasks in the workload that are accurate down to around 1e-8. That may actually be infeasible, but it would be interesting if it were a spur or a challenge to people.

Setting of Accuracy Levels

My feeling is that there should be a way to define this in a way that depends on $n$, but I'm not sure of the maths right now. I'd have to do some thinking, but it feels like the maximum allowed error should grow as a function of $\sqrt{n}$, both for statistical and numerical reasons.

Text I've removed from the specification document:

Underlyings

Ranges of the walk parameters should reflect typical underlyings observed in the market - a good approach would be to pick a whole bunch of actual time-series at different points in time (say 100 underlyings over 10 year-long periods to give 1000 underlyings, or 3000 for a full fit with all thre models), then do a maximum entropy fit. If this doesn't result in any "hard" parameters, then we'd want to revisit that.

Options

We plan to enhance the benchmark for more products later, e.g. to American / Bermudian options, products and models with relationships - products which depend on other products, models that are correlated (e.g. basket options, credit and forex swaps).

The parameters are again inspired by real-world option parameters, though in this case it is less clear how to choose them. There is a reasonable argument for choosing them equally spaced in some sense, rather than driven by traded volume or something, as we don't know what is important to different people.

It is crucial that the provided model parameters reflect realistic scenarios a) of day-to-day situations in different markets (e.g. liquid stock exchange, FOREX, …), but also critical corner cases that have shown to be hard to handle in the past. The data should therefore be legitimated carefully by one or more of the business partners (see below).

We need to ensure that prices end up in the "meaningful" range, so that we don't get prices in the range which could get rounded to zero, or cause problems with relative error. I don't know how to define that right now though :)

My two cents:
We could potentially provide ranges of values for parameters, so that researchers could generate their own problems that are still "sensible". To simulate market pricing conditions, researchers could then also generate products on the fly from the ranges, and characterise how their systems cope with these sorts of problems. We could use this approach to generate the parameters we provide as part of the benchmark at least.