intuition-io / intuition Goto Github PK
View Code? Open in Web Editor NEWQuantitative trading kit, for hackers
Home Page: intuition.io
License: Apache License 2.0
Quantitative trading kit, for hackers
Home Page: intuition.io
License: Apache License 2.0
While a R client and a R server are able to communicate, node.js broker can't read messages sent from R client (./server/shiny-backtest/RClientInterface.R)
11 Mar 04:31:52 - router13901: received from +�n���A0�t��1��N -
undefined:0
^
SyntaxError: Unexpected end of input
at Object.parse (native)
at Socket. (/home/xavier/dev/projects/ppQuanTrade/server/server_fwd.js:63:26)
at Socket.EventEmitter.emit (events.js:106:17)
at Socket._flush._flushing (/usr/lib/node_modules/zmq/lib/index.js:288:25)
at process.startup.processNextTick.process._tickCallback (node.js:245:9)
Plus, socket.options(), documented on https://github.com/armstrtw/rzmq/tree/master/man is not available (impossible to set socket identity, needed for routage).
According to the author zmq devices are not implemented yet, may be a clue for the reading error.
if LANG isn't set (like in a default docker container) :
intuition/utils.py", line 55, in _detect_timezone TypeError: 'NoneType' object has no attribute '__getitem__'
For Live fx trading or backtesting with minutes data , zipline fails as the comparison is between different time frames.
(When i remove benchmark comparison assert )
With live data , algorithm is called for first tick and then system exits ? Is there any way to continuously call specified algorithm ?
Sample logs after for forex live backtest
root@avinash-vaio:/avinash/python-projects/ppQuanTrade/scripts# ./run_backtest.sh
C'est parti...
[2013-03-22 02:13] Database - INFO: Reading NeuronQuant MySQL configuration... (192.168.249.202)
[2013-03-22 02:13] Database - INFO: Reading NeuronQuant MySQL configuration... (192.168.249.202)
[2013-03-22 02:13] ZMQ Messaging - INFO: Signal manager for handling code 2 (192.168.249.202)
[2013-03-22 02:13] Engine - INFO: Configuration is Done. (192.168.249.202)
[2013-03-22 02:13] Engine - INFO:
-- Running backetester...
Using algorithm: BuyAndHold
(192.168.249.202)
[2013-03-22 02:13] Engine - INFO:
-- Using portfolio manager: Constant
(192.168.249.202)
[2013-03-22 02:13] Engine - INFO: [Debug] Algorithm BuyAndHold available, getting a reference to it. (192.168.249.202)
[2013-03-22 02:13] Engine - INFO: [Debug] Manager Constant available, getting a reference and initializing it. (192.168.249.202)
[2013-03-22 02:13] Database - INFO: Reading NeuronQuant MySQL configuration... (192.168.249.202)
[2013-03-22 02:13] Database - INFO: Reading NeuronQuant MySQL configuration... (192.168.249.202)
Benchmark on 2013-03-21 00:00:00+00:00: -0.00828249000776
[2013-03-22 02:13] Forex - DEBUG: [Fri, 22 Mar 2013 02:13:23 GMT] Authentification successful OK:200 (192.168.249.202)
[2013-03-22 02:13] DataLiveSource - DEBUG: Waiting for Forex update (192.168.249.202)
[2013-03-22 02:13] DataLiveSource - DEBUG: New income data, fire an event ! (192.168.249.202)
[2013-03-22 02:13] DataLiveSource - DEBUG: Data available:
Ask.Price 1.29127
Bid.Price 1.29122
High 1.28899
Low 1.29144
TimeStamp 1363918395849
Name: EUR/USD (192.168.249.202)
[2013-03-22 02:13] DataLiveSource - DEBUG: Data available:
Ask.Price 122.591
Bid.Price 122.576
High 122.373
Low 122.704
TimeStamp 1363918404545
Name: EUR/JPY (192.168.249.202)
[2013-03-22 02:13] Performance - INFO: Simulated 1 trading days out of 1. (192.168.249.202)
[2013-03-22 02:13] Performance - INFO: first open: 2013-03-21 13:30:00+00:00 (192.168.249.202)
[2013-03-22 02:13] Performance - INFO: last close: 2013-03-21 20:00:00+00:00 (192.168.249.202)
[2013-03-22 02:13] Trade Labo - INFO: Portfolio returns: 100000.0 (192.168.249.202)
real 0m12.835s
user 0m10.537s
sys 0m0.532s
root@avinash-vaio:/avinash/python-projects/ppQuanTrade/scripts#
I am eagerly waiting to try out various algorthms on live data. It would be nice if you fix live algo run on fx data .
I have problems with install on both Ubuntu 12.04 and Windows.
Could you give some additional info about install process.
The following packages have unmet dependencies:
r-base : Depends: r-base-core (>= 3.0.1-6raring0) but 3.0.1-6precise0 is to be installed
Depends: r-recommended (= 3.0.1-6raring0) but 3.0.1-3precise is to be installed
E: Unable to correct problems, you have held broken packages.
Hi,
Is it possible for you to checkin dashboard code or make shiny webapp more stable. About six months back writing strategies based on minutes data was not supported , is it supported now ?
Was going to clone all the repositories this morning to try intuition for myself, but the intuition-plugins repository is gone. Was really eager to try out the API.
Consider teafiles + lmdb for time series persistence and transactions.
I think a flat file storage system could be a great asset to the project. I have only looked at the teafile spec and have started to use lmdb and Bolt (a golang implementation).
HDF5 seems to be highly regarded in trade apps as well.
I see influxdb in the dev requirements for insight. My problem with influx db for the moment (this could very well change) is that the series have to be sorted after every query it seems.. In a flat file data system we could keep the keys pre sorted by timestamp and write the custom logic to add layers of additional analysis if and when it needs to be persisted with very little overhead to lmdb.
Proof of concept in the works.
/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py:320: Warning: Data truncated for column 'SortinoRatio' at row 1
cursor.execute(statement, parameters)
/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py:320: Warning: Data truncated for column 'Information' at row 1
cursor.execute(statement, parameters)
Takes both value 2005
Without orders during simulation, sharpe ration is infinite (pretty cool !) and other perfs don't make sense.
Therefore propagate the error through myqsl save and client (shiny access)
For ppQuantrade testing purpose i added one test algorithm. When i calculate profit my self i get very good result but when i use self.portfolio object i get very different output. Do you see any issues the way i am using portfolio
This is algorithm
class Test(TradingAlgorithm):
'''
https://www.quantopian.com/posts/this-is-amazing
!! Many transactions, so makes the algorithm explode when traded with many positions
'''
def initialize(self, properties):
self.debug = properties.get('debug', 0)
window_length = properties.get('window_length', 3)
self.purchase_price = 0
self.orderCounter = 0
self.amount = 10000
self.add_transform(SMA, 'mavg', ['price'], window_length=window_length)
self.totalProfit = 0
self.max_notional = 100000.1
self.min_notional = -100000.0
def handle_data(self, data):
''' ---------------------------------------------------------- Init --'''
user_instruction = self.manager.update(self.portfolio, self.datetime.to_pydatetime())
''' ---------------------------------------------------------- Scan --'''
for ticker in data:
sma = data[ticker].mavg
price = data[ticker].price
cash = self.portfolio.cash
notional = self.portfolio.positions[ticker].amount * price
capital_used = self.portfolio.capital_used
if sma * 0.95 > price :
self.logger.info('Cash : {} Notional : {} Capital used {} ' , cash , notional , capital_used)
self.order(ticker, self.amount)
self.logger.info('Cash : {} Notional : {} Capital used {} ' , cash , notional , capital_used)
self.purchase_price += price
self.orderCounter += 1
if self.debug:
self.logger.info('{}: Ordering {} {} stocks @ {} '.format(self.datetime, ticker, self.amount , self.purchase_price / self.orderCounter))
elif self.orderCounter > 0 and self.purchase_price / self.orderCounter < 1.1 * price:
self.logger.info('-Cash : {} Notional : {} Capital used {} ' , cash , notional , capital_used)
self.order(ticker, -self.orderCounter * self.amount)
self.logger.info('-Cash : {} Notional : {} Capital used {} ' , cash , notional , capital_used)
self.totalProfit += (price - self.purchase_price / self.orderCounter) * self.amount * self.orderCounter
if self.debug:
self.logger.info('{}: Selling {} {} stocks @ {} Profit : {}'.format(self.datetime, ticker, -self.orderCounter * self.amount, price , self.totalProfit))
self.purchase_price = 0
self.orderCounter = 0
This is output
[2013-04-14 06:40] Trade Labo - INFO: Portfolio returns: 93729.21875
manually calculated returns : 192000
This is log of executed algorithm
INFO [2013-04-13 23:40:37] python /avinash/python-projects/ppQuanTrade/application/app.py --tickers XIV --algorithm Test --manager Constant --start 2012-07-01 --end 2013-03-31 --exchange nasdaq --database test
[2013-04-14 06:40] INFO: Configuration: Found local configuration file, loading ~/.quantrade/default.json
[2013-04-14 06:40] INFO: Database: Reading QuanTrade MySQL configuration...
[2013-04-14 06:40] INFO: Database: Opening database stock_data
[2013-04-14 06:40] DEBUG: Configuration: Reading commandline arguments
[2013-04-14 06:40] DEBUG: Configuration: Mapping arguments to backtest parameters dictionnary
[2013-04-14 06:40] Configuration - INFO: Reading strategie configuration from json files (192.168.1.6)
[2013-04-14 06:40] Configuration - INFO: Configuration is Done. (192.168.1.6)
[2013-04-14 06:40] Database - INFO: Reading QuanTrade MySQL configuration... (192.168.1.6)
[2013-04-14 06:40] Database - INFO: Opening database stock_data (192.168.1.6)
[2013-04-14 06:40] DataFeed - DEBUG: No matching name, trying symbol list... (192.168.1.6)
[2013-04-14 06:40] Database - INFO: Got infos: <Equity('xiv','VelocityShares Da','PCX', '','','')> (192.168.1.6)
[2013-04-14 06:40] DataFeed - INFO: Retrieving XIV quotes from database (192.168.1.6)
/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py:320: Warning: Incorrect datetime value: '2012-07-01 05:00:00+00:00' for column 'Date' at row 1
cursor.execute(statement, parameters)
/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py:320: Warning: Incorrect datetime value: '2013-03-31 05:00:00+00:00' for column 'Date' at row 1
cursor.execute(statement, parameters)
[2013-04-14 06:40] Engine - INFO:
-- Running backetester...
Using algorithm: Test
(192.168.1.6)
[2013-04-14 06:40] Engine - INFO:
-- Using portfolio manager: Constant
(192.168.1.6)
[2013-04-14 06:40] Engine - INFO: Algorithm Test available, getting a reference to it. (192.168.1.6)
[2013-04-14 06:40] Engine - INFO: Manager Constant available, getting a reference and initializing it. (192.168.1.6)
[2013-04-14 06:40] Database - INFO: Reading QuanTrade MySQL configuration... (192.168.1.6)
[2013-04-14 06:40] Database - INFO: Opening database stock_data (192.168.1.6)
[2013-04-14 06:40] Engine - INFO: Re-loading last Xavier Bruhiere portfolio from database (192.168.1.6)
[2013-04-14 06:40] Manager - INFO: Loading portfolio from database (192.168.1.6)
[2013-04-14 06:40] Engine - INFO: Portfolio setup successful (192.168.1.6)
[2013-04-14 06:40] Transform - INFO: Running StatefulTransform mavg
[2013-04-14 06:40] Test - INFO: Cash : 100000.0 Notional : 0.0 Capital used 0.0 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 100000.0 Notional : 0.0 Capital used 0.0 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2012-07-23 00:00:00+00:00: Ordering XIV 10000 stocks @ 11.8 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 97149.84375 Notional : 2825.0 Capital used -2850.15625 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 97149.84375 Notional : 2825.0 Capital used -2850.15625 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2012-07-24 00:00:00+00:00: Ordering XIV 10000 stocks @ 11.55 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 94274.53125 Notional : 5700.0 Capital used -5725.46875 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 94274.53125 Notional : 5700.0 Capital used -5725.46875 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2012-07-25 00:00:00+00:00: Ordering XIV 10000 stocks @ 11.5 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: -Cash : 91172.8125 Notional : 9225.0 Capital used -8827.1875 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: -Cash : 91172.8125 Notional : 9225.0 Capital used -8827.1875 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2012-07-26 00:00:00+00:00: Selling XIV -30000 stocks @ 12.3 Profit : 24000.0 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 94295.625 Notional : 8000.0 Capital used -5704.375 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 94295.625 Notional : 8000.0 Capital used -5704.375 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2012-09-26 00:00:00+00:00: Ordering XIV 10000 stocks @ 16.0 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: -Cash : 89885.78125 Notional : 13125.0 Capital used -10114.21875 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: -Cash : 89885.78125 Notional : 13125.0 Capital used -10114.21875 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2012-09-27 00:00:00+00:00: Selling XIV -10000 stocks @ 17.5 Profit : 39000.0 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 94101.71875 Notional : 8000.0 Capital used -5898.28125 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 94101.71875 Notional : 8000.0 Capital used -5898.28125 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2012-10-23 00:00:00+00:00: Ordering XIV 10000 stocks @ 16.0 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 90094.375 Notional : 11925.0 Capital used -9905.625 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 90094.375 Notional : 11925.0 Capital used -9905.625 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2012-10-24 00:00:00+00:00: Ordering XIV 10000 stocks @ 15.95 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: -Cash : 85986.40625 Notional : 16300.0 Capital used -14013.59375 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: -Cash : 85986.40625 Notional : 16300.0 Capital used -14013.59375 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2012-10-25 00:00:00+00:00: Selling XIV -20000 stocks @ 16.3 Profit : 46000.0 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 90053.28125 Notional : 11850.0 Capital used -9946.71875 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 90053.28125 Notional : 11850.0 Capital used -9946.71875 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2012-11-07 00:00:00+00:00: Ordering XIV 10000 stocks @ 15.8 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: -Cash : 86121.40625 Notional : 15600.0 Capital used -13878.59375 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: -Cash : 86121.40625 Notional : 15600.0 Capital used -13878.59375 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2012-11-08 00:00:00+00:00: Selling XIV -10000 stocks @ 15.6 Profit : 44000.0 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 89989.53125 Notional : 12900.0 Capital used -10010.46875 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 89989.53125 Notional : 12900.0 Capital used -10010.46875 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2012-12-21 00:00:00+00:00: Ordering XIV 10000 stocks @ 17.2 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: -Cash : 85579.6875 Notional : 17500.0 Capital used -14420.3125 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: -Cash : 85579.6875 Notional : 17500.0 Capital used -14420.3125 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2012-12-24 00:00:00+00:00: Selling XIV -10000 stocks @ 17.5 Profit : 47000.0 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 89671.40625 Notional : 12375.0 Capital used -10328.59375 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 89671.40625 Notional : 12375.0 Capital used -10328.59375 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2012-12-26 00:00:00+00:00: Ordering XIV 10000 stocks @ 16.5 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: -Cash : 85513.125 Notional : 16500.0 Capital used -14486.875 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: -Cash : 85513.125 Notional : 16500.0 Capital used -14486.875 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2012-12-27 00:00:00+00:00: Selling XIV -10000 stocks @ 16.5 Profit : 47000.0 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 89406.09375 Notional : 11775.0 Capital used -10593.90625 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 89406.09375 Notional : 11775.0 Capital used -10593.90625 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2012-12-28 00:00:00+00:00: Ordering XIV 10000 stocks @ 15.7 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: -Cash : 85247.8125 Notional : 16500.0 Capital used -14752.1875 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: -Cash : 85247.8125 Notional : 16500.0 Capital used -14752.1875 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2012-12-31 00:00:00+00:00: Selling XIV -10000 stocks @ 16.5 Profit : 55000.0 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 181275.46875 Notional : -80625.0 Capital used 81275.46875 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 181275.46875 Notional : -80625.0 Capital used 81275.46875 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2013-02-21 00:00:00+00:00: Ordering XIV 10000 stocks @ 21.5 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: -Cash : 175658.125 Notional : -78050.0 Capital used 75658.125 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: -Cash : 175658.125 Notional : -78050.0 Capital used 75658.125 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2013-02-22 00:00:00+00:00: Selling XIV -10000 stocks @ 22.3 Profit : 63000.0 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 180395.78125 Notional : -71625.0 Capital used 80395.78125 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 180395.78125 Notional : -71625.0 Capital used 80395.78125 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2013-02-25 00:00:00+00:00: Ordering XIV 10000 stocks @ 19.1 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 175507.96875 Notional : -67900.0 Capital used 75507.96875 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: Cash : 175507.96875 Notional : -67900.0 Capital used 75507.96875 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2013-02-26 00:00:00+00:00: Ordering XIV 10000 stocks @ 19.25 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: -Cash : 170293.125 Notional : -67275.0 Capital used 70293.125 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: -Cash : 170293.125 Notional : -67275.0 Capital used 70293.125 (192.168.1.6)
[2013-04-14 06:40] Test - INFO: 2013-02-27 00:00:00+00:00: Selling XIV -20000 stocks @ 20.7 Profit : 92000.0 (192.168.1.6)
[2013-04-14 06:40] Transform - INFO: Finished StatefulTransform mavg
[2013-04-14 06:40] Performance - INFO: Simulated 185 trading days out of 185. (192.168.1.6)
[2013-04-14 06:40] Performance - INFO: first open: 2012-07-02 13:30:00+00:00 (192.168.1.6)
[2013-04-14 06:40] Performance - INFO: last close: 2013-03-28 20:00:00+00:00 (192.168.1.6)
[2013-04-14 06:40] Trade Labo - INFO: Portfolio returns: 93729.21875 (192.168.1.6)
/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py:320: Warning: Data truncated for column 'SortinoRatio' at row 1
cursor.execute(statement, parameters)
/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py:320: Warning: Data truncated for column 'Information' at row 1
cursor.execute(statement, parameters)
[2013-04-14 06:40] Trade Labo - INFO:
Returns: -6.27078125% / 15.0766152569%
Volatility: 0.0487673359095
Sharpe: -2.67210000663
Max drawdown: 0.126171024824
(192.168.1.6)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.