groupoid / cubical.systems Goto Github PK

Replaying as a basis for simulation, trading and integrity checking.

Trader should have access to same incoming streams and interoperate with Execution.

Books consistency are being checked by sequence numbers assigned to messages and Order Book snapshots. Replay is a part of both consistency checking and snapshot initialisation.

(for GDAX, trades are available as 'match' events from WS API; for BitMEX, there is 'trade' channel)

All trades should be recorded like this (one file per symbol per 24h day):

timestamp price amount

e.g.

2017-02-27T13:20:33.486Z 1124.54 -5600

(positive amounts are BUY trades, negative are SELL trades).

Extend trade protocol with snapshot requests on stream initialization.
Applicable only for GDAX.

Soon, we'll get out of IDs :)

Instead, internal order book IDs need to be reused.

If an order is removed from the book, its ID can be reused again.

(E.g. by keeping a sorted set of recycled IDs for each side of each book, and getting the first one from here each time we need a new ID)

Remember, BUY trades are mapped to positive amounts, SELL to negative.

Now, all amounts in trade logs are positive :(

E.g. every 10 minutes, or once per 1000 updates, dump current full book status to the data file.

[* 0] [+1 1.1134.05 0.56] [+2 1.1134.07 2.5] [+3 1134.08 0.8] ...

This is necessary to be able to build the book from file in memory, or to recover from errors (losing only the data between the "keyframes").

Add the following instruments for Bitmex (all of them are futures or swaps, so currency pair naming conventions do not apply).

COIN_BH17
DASH7D
ETH7D
XBTUSD

(futures have changing symbols, so they are to be handled separately)

In our storage, they are to be mapped as is with '@BitMEX' added as exchange designator, e.g.

COIN_BH17@BitMEX
DASH7D@BitMEX
ETH7D@BitMEX
XBTUSD@BitMEX

While I design FIX API, let's do something simple with JSON:

{"type": "subscribe", "channel": ["trade", "orderBookL2"], "sources": ["gdax", "bitmex"]}

Order book update ("market data"):

< {"type": "book", "source": "gdax", "symbol": "BTC/USD", "updates": [[232, 1134.23, 0.5], [-23, 0], [-322, 1132.1, 7.5]]}

(full book is encoded with "remove all" update, then list of add commands for all entries in the book)

< {"type": "trade", "source": "gdax", "symbol": "BTC/USD", "price": 1183.12, "amount": -2354.5}

etc.

• Book API — done
• Trader/Execution API — awaiting
• Alko API — awaiting

Let's change BID order ids in the book to be negative, and OFFER to be positive. This will get rid of the ugly "+0/-0" shit.

E.g.

[+435 1134.0 0.43] -- add/replace offers with id 435, price 1134.0 amount 0.43
[-245 0] -- remove bid order with id 245 from the book