Desired Behavior

I want to be able to submit/merge PRs with some basic guarantee of compilation

Impact

The desired behavior will allow me to ensure that my build always compiles

Feature Suggestion

We can achieve the desired behavior by using Travis CI

// see a sample bug report here: #1

Describe the bug

// A succinct description of the buggy behavior.
// This section includes what the bot is doing and why you think it is incorrect.

Buysell puts orders at totally predictable intervals. In the extreme case, lets say you use a 5 minute interval. Anyone smart will then put in his order 10 seconds before the orders up date so he can get filled on the right side of the market, e.g., if he knows the price is going up by more than the spread, he'll put in a buy order. Thus, by timing it, our attacker ALWAYS MAKES MONEY. However, if the interval is random, this attack is defeated.

Expected behavior

// A succinct description of what you expected to happen.

random interval for placing new order

Frequency

// How often does this happen?
// Unpredictable = it's hard to predict when it will happen
// Sometimes = it happens every now and then but there's no pattern to it
// Always = it always happens consistently at the same point

always a problem.

Steps To Reproduce

// A concise description of the steps to reproduce the buggy behavior along with screenshots wherever applicable:
n/a

Possible Solution

// Optional, suggest a fix.

add a MAX_TICK_DELAY_SECONDS config field in the trader.cfg file which will produce a random value that will be added to the current TICK_INTERVAL_SECONDS

Your Environment

// Run kelp version to get the version and build information and paste it here
// version: v1.0.0-rc1
// git hash: 1bcc4e9
// build date: 20180813T013016Z
// GOOS: linux
// GOARCH: amd64

...

Context

// A succinct description of how has this bug has affected you or prevented you from accomplishing what you wanted.

I lose money if this bug isn't fixed.

Attachments

// Attach any relevant configuration files, logs, tx hashes, etc. here.

Desired Behavior

I want to be able to cancel some inside offers in my level provider without having the bot modify all the offers and cancel the last offer. The bot should just cancel the inside offers.

Impact

The desired behavior will lower my fee when submitting transactions since it will have fewer operations.

Feature Suggestion

We can achieve the desired behavior by augmenting the state transition logic in sellSideStrategy that converts levels to operations.

Additional Context

This is a follow-up to #46

// see a sample bug report here: #1

Describe the bug

// A succinct description of the buggy behavior.
// This section includes what the bot is doing and why you think it is incorrect.

I trade 4 different markets using the same account number to place the trades. So since you have 4 different kelps running you are going to get an invalid sequence number error.

Expected behavior

// A succinct description of what you expected to happen.

When kelp wakes up, and is ready to do it's thing, it should ideally re-fetch the sequence number. I guess this is a performance optimization?

Frequency

// How often does this happen?
// Unpredictable = it's hard to predict when it will happen
// Sometimes = it happens every now and then but there's no pattern to it
// Always = it always happens consistently at the same point

all the time

Steps To Reproduce

// A concise description of the steps to reproduce the buggy behavior along with screenshots wherever applicable:

run >1 kelp instance with the same SOURCE_SECRET_SEED

Possible Solution

// Optional, suggest a fix.

This bug can be avoided by using unique SOURCE_SECRET_SEED for each kelp instance.

You cannot stagger the start time so no overlap since kelp doesn't fetch the seq number each time.

Your Environment

// Run kelp version to get the version and build information and paste it here
// version: v1.0.0-rc1
// git hash: 1bcc4e9
// build date: 20180813T013016Z
// GOOS: linux
// GOARCH: amd64

n/a

Context

// A succinct description of how has this bug has affected you or prevented you from accomplishing what you wanted.

n/a

Attachments

// Attach any relevant configuration files, logs, tx hashes, etc. here.

Describe the bug

After running kelp for 1 day, we are seeing a weird bug

2018/10/17 18:32:16 error decoding horizon.Problem: EOF
2018/10/17 18:32:16 Can't load offers: error decoding horizon.Problem: EOF
2018/10/17 18:32:16 error decoding horizon.Problem: EOF
2018/10/17 18:32:16 Can't load offers: error decoding horizon.Problem: EOF
2018/10/17 18:32:16 error: cannot load offers to compute liabilities for asset (native): error decoding horizon.
Problem: EOF
2018/10/17 18:32:16 liabilities after resetting
2018/10/17 18:32:16 Can't load offers: error decoding horizon.Problem: EOF
2018/10/17 18:32:16 error: cannot load offers to compute liabilities for asset (native): error decoding horizon.
Problem: EOF
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x75a9bb]
goroutine 1 [running]:
github.com/lightyeario/kelp/plugins.(*SDEX).logLiabilities(0xc420154120, 0x8b18ad, 0x6, 0x0, 0x0, 0x0, 0x0)
        /Users/nikhilsaraf/dev/go/src/github.com/lightyeario/kelp/plugins/sdex.go:435 +0xfb
github.com/lightyeario/kelp/plugins.(*SDEX).LogAllLiabilities(0xc420154120, 0x8b18ad, 0x6, 0x0, 0x0, 0x0, 0x0, 0
x8b83f4, 0x10, 0xc42023cc28, ...)
        /Users/nikhilsaraf/dev/go/src/github.com/lightyeario/kelp/plugins/sdex.go:443 +0x65
github.com/lightyeario/kelp/trader.(*Trader).update(0xc42011fc48)
        /Users/nikhilsaraf/dev/go/src/github.com/lightyeario/kelp/trader/trader.go:105 +0x1e9
github.com/lightyeario/kelp/trader.(*Trader).Start(0xc42011fc48)
        /Users/nikhilsaraf/dev/go/src/github.com/lightyeario/kelp/trader/trader.go:67 +0x6f
github.com/lightyeario/kelp/cmd.init.4.func1(0xb447e0, 0xc420061740, 0x0, 0x6)
        /Users/nikhilsaraf/dev/go/src/github.com/lightyeario/kelp/cmd/trade.go:109 +0x9d2
github.com/lightyeario/kelp/vendor/github.com/spf13/cobra.(*Command).execute(0xb447e0, 0xc4200616e0, 0x6, 0x6, 0
xb447e0, 0xc4200616e0)
        /Users/nikhilsaraf/dev/go/src/github.com/lightyeario/kelp/vendor/github.com/spf13/cobra/command.go:702 +
0x2c6
github.com/lightyeario/kelp/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xb44e40, 0x403dfc, 0xc42006a058, 
0x0)
        /Users/nikhilsaraf/dev/go/src/github.com/lightyeario/kelp/vendor/github.com/spf13/cobra/command.go:783 +
0x2e4
github.com/lightyeario/kelp/vendor/github.com/spf13/cobra.(*Command).Execute(0xb44e40, 0x0, 0x7b1582)
        /Users/nikhilsaraf/dev/go/src/github.com/lightyeario/kelp/vendor/github.com/spf13/cobra/command.go:736 +
0x2b
main.main()
        /Users/nikhilsaraf/dev/go/src/github.com/lightyeario/kelp/main.go:10 +0x2d

Expected behavior

It might be possible that horizon was not responding or some library is missing.
Restarting the bot worked fine though.

Frequency

The frequency is: Unpredictable

Steps To Reproduce

It is very rare issue, We can give run log if it is required.
Here are the steps to reproduce the issue (see attachments in section below):

1. Set up your configuration file with these parameters:

2018/10/16 07:36:55 Starting Kelp Trader: v1.0.0 [e2c473c2e866a9b27d485c403a14a4e286b51c7d]
2018/10/16 07:36:55 configs:
2018/10/16 07:36:55      SOURCE_SECRET_SEED: 
2018/10/16 07:36:55      TRADING_SECRET_SEED: [secret key to account GDGYAYBTNY6CXA77EJ2PTBJB3DR2MYFOE57I2GZYIH7
FGOHSPI4XGDVD]
2018/10/16 07:36:55      ASSET_CODE_A: XLM
2018/10/16 07:36:55      ISSUER_A: 
2018/10/16 07:36:55      ASSET_CODE_B: PEDI
2018/10/16 07:36:55      ISSUER_B: GBVUDZLMHTLMZANLZB6P4S4RYF52MVWTYVYXTQ2EJBPBX4DZI2SDOLLY
2018/10/16 07:36:55      TICK_INTERVAL_SECONDS: 300
2018/10/16 07:36:55      HORIZON_URL: https://horizon.stellar.org
2018/10/16 07:36:55 Trading XLM: for PEDI:GBVUDZLMHTLMZANLZB6P4S4RYF52MVWTYVYXTQ2EJBPBX4DZI2SDOLLY
2018/10/16 07:36:55 Current strategy: balanced
2018/10/16 07:36:55 Using network passphrase: Public Global Stellar Network ; September 2015
2018/10/16 07:36:55 No Source Account Set
2018/10/16 07:36:55 configs:
2018/10/16 07:36:55      PRICE_TOLERANCE: 0.001
2018/10/16 07:36:55      AMOUNT_TOLERANCE: 0.001
2018/10/16 07:36:55      SPREAD: 0.01
2018/10/16 07:36:55      MIN_AMOUNT_SPREAD: 0.0005
2018/10/16 07:36:55      MAX_AMOUNT_SPREAD: 0.0005
2018/10/16 07:36:55      MAX_LEVELS: 10
2018/10/16 07:36:55      LEVEL_DENSITY: 0.2
2018/10/16 07:36:55      ENSURE_FIRST_N_LEVELS: 4
2018/10/16 07:36:55      MIN_AMOUNT_CARRYOVER_SPREAD: 0.01
2018/10/16 07:36:55      MAX_AMOUNT_CARRYOVER_SPREAD: 0.01
2018/10/16 07:36:55      CARRYOVER_INCLUSION_PROBABILITY: 1
2018/10/16 07:36:55      VIRTUAL_BALANCE_BASE: 0
2018/10/16 07:36:55      VIRTUAL_BALANCE_QUOTE: 0
2018/10/16 07:36:55 ----------------------------------------------------

2. Start the bot using this command: ... ./kelp trade --botConf sample_trader.cfg --strategy balanced --stratConf sample_balanced.cfg
   ...

Possible Solution

No idea

Your Environment

version: v1.0.0
git hash: e2c473c
build date: 20181015T211324Z
GOOS: linux
GOARCH: amd64

Context

I am unable to run this bot 24x7 because of this bug as the kelp bot is exiting.

Attachments

If run log is required, I can provide that. If there is any other log inside kelp, let me know. I will add it here.

// see a sample bug report here: #1

Describe the bug

kelp crashes after running for 2 weeks because horizon is down.

2018/10/30 15:27:44 error decoding horizon.Problem: EOF
2018/10/30 15:27:44 Can't load offers: error decoding horizon.Problem: EOF
2018/10/30 15:27:44 error decoding horizon.Problem: EOF
2018/10/30 15:27:44 Can't load offers: error decoding horizon.Problem: EOF
2018/10/30 15:27:44 error: cannot load offers to compute liabilities for asset (USD:GDSRCV5VTM3U7Y3L6DFRP3PEGBNQMGOWSRTGSBWX6Z3H6C7JHRI4XFJP): error decoding horizon.Problem: EOF
2018/10/30 15:27:44 liabilities after resetting
2018/10/30 15:27:44 Can't load offers: error decoding horizon.Problem: EOF
2018/10/30 15:27:44 error: cannot load offers to compute liabilities for asset (USD:GDSRCV5VTM3U7Y3L6DFRP3PEGBNQMGOWSRTGSBWX6Z3H6C7JHRI4XFJP): error decoding horizon.Problem: EOF
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x75a88b]

Expected behavior

// A succinct description of what you expected to happen.
This happened before on 10/17. It happened to me again on 10/30. why not just catch the error and try again the next interval?

Frequency

The frequency is: sometimes

Steps To Reproduce

start with the buysell and let it run for 10 days

Possible Solution

// Optional, suggest a fix.

This bug can be fixed by ...

Your Environment

// Run kelp version to get the version and build information and paste it here
stk@steve-kelp:~/kelp$ ./kelp version
version: v1.0.0-rc3
git hash: c8363e7
build date: 20180929T155837Z
GOOS: linux
GOARCH: amd64

...

Context

// A succinct description of how has this bug has affected you or prevented you from accomplishing what you wanted.

I am unable to keep trading

Attachments

// Attach any relevant configuration files, logs, tx hashes, etc. here.
logfile.txt

Describe the bug

If the bot was to crash with a panic then it is not logged to the file when using the --log option

Expected behavior

I think the correct behavior in this situation is for the bot to log these errors to the file as well so we can fully depend on the log files

Frequency

The frequency is: Always

Steps To Reproduce

Here are the steps to reproduce the issue (see attachments in section below):

1. Create a strategy with a panic operation to force it to crash
2. Start the bot using the --log flag
3. the bot will crash and the standard out will display the panic but the log file will not.

Possible Solution

This bug can be fixed by catching any panics at the highest level and logging that before exiting.

Your Environment

git hash: 8e6fae2

Desired Behavior

I want to reduce the jitter in the orderbook caused by unreachable price feeds. This will allow me to configure the bot in a way that prevents deletion of all offers in such circumstances to a configurable threshold.

Impact

The desired behavior will allow me to run the bot and make more smooth markets.

Feature Suggestion

We can achieve the desired behavior by allowing upto a max of N continuous failed iterations before deciding to delete all offers. This applies only to the case where we delete offers without exiting.

Describe the bug

The bot tries to divide by zero in the sellSideStrategy when the price feed for ASSET_B fails, which caused the bot to crash.

Expected behavior

I think the correct behavior in this situation is for the bot to skip the update cycle if any one of the price feeds fail or if either of the price feeds return a zero value.

Frequency

The frequency is: Sometimes [when the price feeds return an invalid value]

Steps To Reproduce

Here are the steps to reproduce the issue (see attachments in section below):

1. Run the bot with price feeds
2. Return a zero value from the price feed (can use the "fixed" value to test this)
3. bot will crash with the divide by 0 error

Possible Solution

See expected behavior

Your Environment

version: v1.0.0-rc1
git hash: 337f478b0e7a1dc235aef31788754bc1ab11b6a1
build date: 20180813T231816Z
GOOS: linux
GOARCH: amd64

Context

I am unable to run the bot reliably because of this bug.

Attachments

See the log here

2018/08/26 11:11:35 error: center price couldn't be loaded! | Get net/http: request canceled (Client.Timeout exceeded while awaiting headers)
2018/08/26 11:11:35 levels couldn't be loaded: Get net/http: request canceled (Client.Timeout exceeded while awaiting headers)
2018/08/26 11:11:43 center price: 0.0000000
2018/08/26 11:11:43 error in buying sub-strategy: Get request canceled (Client.Timeout exceeded while awaiting headers)
2018/08/26 11:11:43 created 0 operations to delete offers
2018/08/26 11:11:43 sleeping for  seconds...
2018/08/26 11:13:43 ----------------------------------------------------------------------------------------------------
2018/08/26 11:13:43 created 0 operations to prune excess offers
2018/08/26 11:13:43 sell,create,p=+Inf,a=0.0000000
2018/08/26 11:13:43 error: cannot place sell order, zero amount
2018/08/26 11:13:43 sell,create,p=+Inf,a=0.0000000
2018/08/26 11:13:43 error: cannot place sell order, zero amount
2018/08/26 11:13:43 sell,create,p=+Inf,a=0.0000000
2018/08/26 11:13:43 error: cannot place sell order, zero amount
2018/08/26 11:13:43 sell,create,p=+Inf,a=0.0000000
2018/08/26 11:13:43 error: cannot place sell order, zero amount
2018/08/26 11:13:43 sell,create,p=+Inf,a=0.0000000
2018/08/26 11:13:43 error: cannot place sell order, zero amount
2018/08/26 11:13:43 sell,create,p=+Inf,a=0.0000000
2018/08/26 11:13:43 error: cannot place sell order, zero amount
2018/08/26 11:13:43 sell,create,p=0.0000000,a=23619.2878901
panic: division by zero

goroutine 1 [running]:
math/big.(*Rat).SetFrac64(0xc42031c440, 0x0, 0x0, 0x0)
  /usr/local/Cellar/go/1.10.3/libexec/src/math/big/rat.go:311 +0xfb
math/big.NewRat(0x0, 0x0, 0xc42061f000)
  /usr/local/Cellar/go/1.10.3/libexec/src/math/big/rat.go:25 +0x4f
github.com/lightyeario/kelp/vendor/github.com/stellar/go/xdr.(*Price).String(0xc4201142a0, 0x0, 0x0)
  ~/go/src/github.com/lightyeario/kelp/vendor/github.com/stellar/go/xdr/price.go:9 +0x37
github.com/lightyeario/kelp/plugins.(*sellSideStrategy).UpdateWithOps(0xc420140a10, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
  ~/go/src/github.com/lightyeario/kelp/plugins/sellSideStrategy.go:97 +0xec
github.com/lightyeario/kelp/plugins.(*composeStrategy).UpdateWithOps(0xc4202aec90, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xb73f58, 0x0, 0x0, ...)
  ~/go/src/github.com/lightyeario/kelp/plugins/composeStrategy.go:79 +0x107
github.com/lightyeario/kelp/trader.(*Trader).update(0xc4201f5c48)
  ~/go/src/github.com/lightyeario/kelp/trader/trader.go:120 +0x34a
github.com/lightyeario/kelp/trader.(*Trader).Start(0xc4201f5c48)
  ~/go/src/github.com/lightyeario/kelp/trader/trader.go:65 +0x6f
github.com/lightyeario/kelp/cmd.init.4.func1(0xb50e20, 0xc420069440, 0x0, 0x6)
  ~/go/src/github.com/lightyeario/kelp/cmd/trade.go:109 +0x8fa
github.com/lightyeario/kelp/vendor/github.com/spf13/cobra.(*Command).execute(0xb50e20, 0xc4200693e0, 0x6, 0x6, 0xb50e20, 0xc4200693e0)
  ~/go/src/github.com/lightyeario/kelp/vendor/github.com/spf13/cobra/command.go:702 +0x2c6
github.com/lightyeario/kelp/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xb507c0, 0x403dfc, 0xc420072058, 0x0)
  ~/go/src/github.com/lightyeario/kelp/vendor/github.com/spf13/cobra/command.go:783 +0x2e4
github.com/lightyeario/kelp/vendor/github.com/spf13/cobra.(*Command).Execute(0xb507c0, 0x0, 0x7b66a8)
  ~/go/src/github.com/lightyeario/kelp/vendor/github.com/spf13/cobra/command.go:736 +0x2b
main.main()
  ~/go/src/github.com/lightyeario/kelp/main.go:10 +0x2d

Desired Behavior

Currently, if the balanced strategy has LEVEL_DENSITY < 1.0 it will modify its orders to re-randomize the levels even if no offers have been taken.

I want to have the balanced strategy leave all levels in place if no trades have happened since the last refresh; I don't think re-randomizing is useful when nothing has changed.

Impact

The current behavior submits unnecessary operations to the network. This change will increase efficiency and reduce user fee spending.

Feature Suggestion

We can implement this by having the bot stop modifying offers if the first offer is still in place and within tolerances. I'm submitting a pull request shortly.

Desired Behavior

I want to avoid placing modify orders for all levels if I replenish the inside level

Impact

The desired behavior will allow me to be more efficient with my operations to the exchange

Feature Suggestion

We can achieve the desired behavior by having the sellSideStrategy check if it needs to place an order inside instead of modifying all offers

Describe the bug

The bot is unable to increase the amounts on a market if it has already hit it's capacity limits

Expected behavior

I think the correct behavior in this situation is for the bot to handle this smoothly.

Frequency

The frequency is: Always (in the above scenario)

Steps To Reproduce

Here are the steps to reproduce the issue (see attachments in section below):

1. Start the bot with the buysell strategy with an amount that will max out the lumens it is selling
2. Update the configuration to further increase the amount (AMOUNT_OF_A_BASE)
3. The bot will have op_underfunded errors in the transaction result

Possible Solution

This bug can be fixed by prepending modify offers that reduce it's outstanding liability -- we already do this when the bot deletes offers. Order of the update operations matter.

Your Environment

  version: v1.1.1-7-g880b1b1d
  git hash: 880b1b1d4f99b104587a1a12390bc35ccab03ade
  build date: 20181030T164726Z
  GOOS: linux
  GOARCH: amd64

Context

I am unable to adjust the bot's amounts as desired because of this bug.

Attachments

2018/10/30 09:48:45 Starting the trader bot...
2018/10/30 09:48:45 ----------------------------------------------------------------------------------------------------
2018/10/30 09:48:45  (base) assetA=native, maxA=14194.9999875, trustA=math.MaxFloat64
2018/10/30 09:48:45 (quote) assetB=TEST:GCFFRPDGJHMTMP5WAVEUSDIYNWCDUZTRE2QMLYVVPAYYFKZPMF4AMNTG, maxB=99998999.2296317, trustB=922337203685.4775391
2018/10/30 09:48:46 liabilities after resetting
2018/10/30 09:48:46 asset = {native  }, buyingLiabilities=0.0000000, sellingLiabilities=0.0000000
2018/10/30 09:48:46 asset = {credit_alphanum4 TEST GCFFRPDGJHMTMP5WAVEUSDIYNWCDUZTRE2QMLYVVPAYYFKZPMF4AMNTG}, buyingLiabilities=0.0000000, sellingLiabilities=0.0000000
2018/10/30 09:48:47 price from exchange feed (ccxt-binance/XLM/USDT): bidPrice=0.2244000, askPrice=0.2246000, centerPrice=0.2245000
2018/10/30 09:48:47 feedPair prices: feedA=1.0000000, feedB=0.2245000; centerPrice=4.4543430
2018/10/30 09:48:47 price from exchange feed (ccxt-binance/XLM/USDT): bidPrice=0.2244000, askPrice=0.2246000, centerPrice=0.2245000
2018/10/30 09:48:47 feedPair prices: feedA=0.2245000, feedB=1.0000000; centerPrice=0.2245000
2018/10/30 09:48:47 offer | buy  | level=1 | curPriceQuote=0.2244389 | curAmtBase=10000.0000000 | pruning=false
2018/10/30 09:48:47 offer | buy  | level=2 | curPriceQuote=0.2243828 | curAmtBase=10000.0000000 | pruning=false
2018/10/30 09:48:47 offer | buy  | level=3 | curPriceQuote=0.2243268 | curAmtBase=10000.0000001 | pruning=false
2018/10/30 09:48:47 offer | sell | level=1 | curPriceQuote=0.2245511 | curAmtBase=10000.0000000 | pruning=false
2018/10/30 09:48:47 offer | sell | level=2 | curPriceQuote=0.2246072 | curAmtBase=4170.4999871 | pruning=false
2018/10/30 09:48:47 created 0 operations to prune excess offers
2018/10/30 09:48:48 liabilities after resetting
2018/10/30 09:48:48 asset = {native  }, buyingLiabilities=0.0000000, sellingLiabilities=0.0000000
2018/10/30 09:48:48 asset = {credit_alphanum4 TEST GCFFRPDGJHMTMP5WAVEUSDIYNWCDUZTRE2QMLYVVPAYYFKZPMF4AMNTG}, buyingLiabilities=0.0000000, sellingLiabilities=0.0000000
2018/10/30 09:48:48 buy  | modify | level=1 | targetPriceQuote=0.2244439 | targetAmtBase=12000.0000000 | curPriceQuote=0.2244389 | lowPriceQuote=0.2244439 | highPriceQuote=0.2244439 | curAmtBase=10000.0000000 | minAmtBase=11400.2538191 | maxAmtBase=12600.2805369
2018/10/30 09:48:49 buy  | modify | level=2 | targetPriceQuote=0.2243878 | targetAmtBase=11999.9999999 | curPriceQuote=0.2243828 | lowPriceQuote=0.2243878 | highPriceQuote=0.2243878 | curAmtBase=10000.0000000 | minAmtBase=11400.2537556 | maxAmtBase=12600.2804667
2018/10/30 09:48:49 buy  | modify | level=3 | targetPriceQuote=0.2243318 | targetAmtBase=12000.0000000 | curPriceQuote=0.2243268 | lowPriceQuote=0.2243317 | highPriceQuote=0.2243318 | curAmtBase=10000.0000001 | minAmtBase=11400.2539481 | maxAmtBase=12600.2806794
2018/10/30 09:48:49 sell | modify | level=1 | targetPriceQuote=0.2245561 | targetAmtBase=12000.0000000 | curPriceQuote=0.2245511 | lowPriceQuote=0.2245561 | highPriceQuote=0.2245561 | curAmtBase=10000.0000000 | minAmtBase=11400.0000000 | maxAmtBase=12600.0000000
2018/10/30 09:48:49 sell | modify | level=2 | targetPriceQuote=0.2246123 | targetAmtBase=12000.0000000 | curPriceQuote=0.2246072 | lowPriceQuote=0.2246123 | highPriceQuote=0.2246123 | curAmtBase=4170.4999871 | minAmtBase=11400.0000000 | maxAmtBase=12600.0000000
2018/10/30 09:48:49 we will oversell the asset 'native', amountSelling = 12000.0000000, bal = 14194.9999875, minAccountBal = 24.0000000, liabilities.Selling = 12000.0000000
2018/10/30 09:48:49 computed remainder amount, constrained by selling capacity, returning sellingAmount=2170.9999875, buyingAmount=487.6333005
2018/10/30 09:48:49 sell | modify | level=2 | targetPriceQuote=0.2246123 | targetAmtBase=2170.9999875 | curPriceQuote=0.2246072 | lowPriceQuote=0.2246123 | highPriceQuote=0.2246123 | curAmtBase=4170.4999871 | minAmtBase=11400.0000000 | maxAmtBase=12600.0000000
2018/10/30 09:48:49 liabilities at the end of a call to UpdateWithOps
2018/10/30 09:48:49 asset = {native  }, buyingLiabilities=36000.0000000, sellingLiabilities=14170.9999875
2018/10/30 09:48:49 asset = {credit_alphanum4 TEST GCFFRPDGJHMTMP5WAVEUSDIYNWCDUZTRE2QMLYVVPAYYFKZPMF4AMNTG}, buyingLiabilities=3182.3065005, sellingLiabilities=8077.9613362
2018/10/30 09:48:49 created 5 operations to update existing offers
2018/10/30 09:48:49 reloading sequence number
2018/10/30 09:48:50 tx XDR: AAAAAIpYvGZJ2TY/tgVJSQ0YbYQ6ZnEmoMXitXgxgqsvYXgGAAAB9AAAMKUAAAD6AAAAAAAAAAAAAAAFAAAAAQAAAAC+B05WMqwKNPp1zsO8wisjLSFafnHK96h/lpSKbpq9fgAAAAMAAAABVEVTVAAAAACKWLxmSdk2P7YFSUkNGG2EOmZxJqDF4rV4MYKrL2F4BgAAAAAAAAAGRVkIRwFT7KMATEtAAAAAAAAATscAAAABAAAAAL4HTlYyrAo0+nXOw7zCKyMtIVp+ccr3qH+WlIpumr1+AAAAAwAAAAFURVNUAAAAAIpYvGZJ2TY/tgVJSQ0YbYQ6ZnEmoMXitXgxgqsvYXgGAAAAAAAAAAZE8lcRAVQCYwBMS0AAAAAAAABPeAAAAAEAAAAAvgdOVjKsCjT6dc7DvMIrIy0hWn5xyveof5aUim6avX4AAAADAAAAAVRFU1QAAAAAili8ZknZNj+2BUlJDRhthDpmcSagxeK1eDGCqy9heAYAAAAAAAAABkSLtVoCqDBFAJiWgAAAAAAAAE+JAAAAAQAAAAC+B05WMqwKNPp1zsO8wisjLSFafnHK96h/lpSKbpq9fgAAAAMAAAAAAAAAAVRFU1QAAAAAili8ZknZNj+2BUlJDRhthDpmcSagxeK1eDGCqy9heAYAAAAb8I6wAAAiQ7kAmJaAAAAAAAAATlwAAAABAAAAAL4HTlYyrAo0+nXOw7zCKyMtIVp+ccr3qH+WlIpumr1+AAAAAwAAAAAAAAABVEVTVAAAAACKWLxmSdk2P7YFSUkNGG2EOmZxJqDF4rV4MYKrL2F4BgAAAAUOBE8DACJF6wCYloAAAAAAAABOXQAAAAAAAAACL2F4BgAAAECuKtsmrZqRdESetBUOlQbGmCoHIedy+PcRDYCGZ5JScKkUR6vbwzVvWpNdhNKZAm5KcKQtWf19wUKnq0FkefkCbpq9fgAAAEAGpI2oR8pJYXA5Opf4lyJ71qV0qy4okpB1Fd/Hkhl9fsP2kRk2tYWBOVfuEoVxC9Dj5g8CqUJP3VEwIqDf8IgC
2018/10/30 09:48:50 submitting tx XDR to network (async)
2018/10/30 09:48:50 sleeping for 10 seconds...
2018/10/30 09:48:53 (async) error: result code details: tx code = tx_failed , opcodes = [op_success op_success op_success op_underfunded op_success]

Describe the bug

As of 05:43 CST this morning, 11/21/2018, my Kelp bot is returning error decoding horizon.Problem: EOF for nearly every attempted horizon call. Immediately before the consistent failure a tx_bad_seq error occurred. The first errored out cycle log was:

2018/11/21 05:43:16 (async) error: tx_bad_seq, setting flag to reload seq number
2018/11/21 05:43:16 (async) error: result code details: tx code = tx_bad_seq , opcodes = []
2018/11/21 05:43:16 ----------------------------------------------------------------------------------------------------
2018/11/21 05:43:16 sleeping for 4m47.670314655s...
2018/11/21 05:48:04 intervalTimeController tickInterval=5m0s, shouldUpdate=true, elapsedSinceUpdate=5m0.000081616s
2018/11/21 05:49:03 error decoding horizon.Problem: EOF
2018/11/21 05:50:03 Can't load offers: error decoding horizon.Problem: EOF
2018/11/21 05:50:03 error decoding horizon.Problem: EOF
2018/11/21 05:51:03 Can't load offers: error decoding horizon.Problem: EOF
2018/11/21 05:51:03 error: cannot load offers to compute liabilities for asset (SLT:GCKA6K5PCQ6PNF5RQBF7PQDJWRHO6UOGFMRLK3DYHDOI244V47XKQ4GP): error decoding horizon.Problem: EOF
2018/11/21 05:51:03 liabilities after resetting
2018/11/21 05:52:03 Can't load offers: error decoding horizon.Problem: EOF
2018/11/21 05:52:03 error: cannot load offers to compute liabilities for asset (SLT:GCKA6K5PCQ6PNF5RQBF7PQDJWRHO6UOGFMRLK3DYHDOI244V47XKQ4GP): error decoding horizon.Problem: EOF
2018/11/21 05:52:03 could not fetch liability for asset 'base  ', error = error decoding horizon.Problem: EOF
2018/11/21 05:53:03 Can't load offers: error decoding horizon.Problem: EOF
2018/11/21 05:53:03 error: cannot load offers to compute liabilities for asset (native): error decoding horizon.Problem: EOF
2018/11/21 05:53:03 could not fetch liability for asset 'quote ', error = error decoding horizon.Problem: EOF
2018/11/21 05:53:03 error decoding horizon.Problem: EOF
2018/11/21 05:53:03 deleting all offers
2018/11/21 05:53:03 created 39 operations to delete offers
2018/11/21 05:53:03 reloading sequence number
2018/11/21 05:54:03 error getting seq num: load account failed: error decoding horizon.Problem: EOF
2018/11/21 05:54:03 tx XDR: AAAAAFEaM7CTwKol9+ZL7WdPTrZNPY0g0gWj8WWIZI9mhpEOAAAPPAEgurgAABVEAAAAAAAAAAAAAAAnAAAAAAAAAAMAAAABU0xUAAAAAACUDyuvFDz2l7GAS/fAabRO71HGKyK1bHg43I1zlefuqAAAAAAAAAAAAAAAAA0gRokAmJaAAAAAAALLlCgAAAAAAAAAAwAAAAFTTFQAAAAAAJQPK68UPPaXsYBL98BptE7vUcYrIrVseDjcjXOV5+6oAAAAAAAAAAAAAAAABqRmqwBMS0AAAAAAAsuUJgAAAAAAAAADAAAAAVNMVAAAAAAAlA8rrxQ89pexgEv3wGm0Tu9RxisitWx4ONyNc5Xn7qgAAAAAAAAAAAAAAAANUwPBAJiWgAAAAAACy5QpAAAAAAAAAAMAAAABU0xUAAAAAACUDyuvFDz2l7GAS/fAabRO71HGKyK1bHg43I1zlefuqAAAAAAAAAAAAAAAAAGuOlkAExLQAAAAAALLlCoAAAAAAAAAAwAAAAFTTFQAAAAAAJQPK68UPPaXsYBL98BptE7vUcYrIrVseDjcjXOV5+6oAAAAAAAAAAAAAAAAAIoWCwAGGoAAAAAAAsuUJwAAAAAAAAADAAAAAVNMVAAAAAAAlA8rrxQ89pexgEv3wGm0Tu9RxisitWx4ONyNc5Xn7qgAAAAAAAAAAAAAAAAGzap1AExLQAAAAAACy5QrAAAAAAAAAAMAAAABU0xUAAAAAACUDyuvFDz2l7GAS/fAabRO71HGKyK1bHg43I1zlefuqAAAAAAAAAAAAAAAAANpcqcAJiWgAAAAAALLm6sAAAAAAAAAAwAAAAFTTFQAAAAAAJQPK68UPPaXsYBL98BptE7vUcYrIrVseDjcjXOV5+6oAAAAAAAAAAAAAAAAA26yWwAmJaAAAAAAAsubrAAAAAAAAAADAAAAAVNMVAAAAAAAlA8rrxQ89pexgEv3wGm0Tu9RxisitWx4ONyNc5Xn7qgAAAAAAAAAAAAAAAAG5/YLAExLQAAAAAACy5utAAAAAAAAAAMAAAABU0xUAAAAAACUDyuvFDz2l7GAS/fAabRO71HGKyK1bHg43I1zlefuqAAAAAAAAAAAAAAAAA3aih0AmJaAAAAAAALLm64AAAAAAAAAAwAAAAFTTFQAAAAAAJQPK68UPPaXsYBL98BptE7vUcYrIrVseDjcjXOV5+6oAAAAAAAAAAAAAAAABv8O4wBMS0AAAAAAAsubrwAAAAAAAAADAAAAAVNMVAAAAAAAlA8rrxQ89pexgEv3wGm0Tu9RxisitWx4ONyNc5Xn7qgAAAAAAAAAAAAAAAAOBU47AJiWgAAAAAACy5uwAAAAAAAAAAMAAAABU0xUAAAAAACUDyuvFDz2l7GAS/fAabRO71HGKyK1bHg43I1zlefuqAAAAAAAAAAAAAAAAAC0i1cAB6EgAAAAAALLm7EAAAAAAAAAAwAAAAFTTFQAAAAAAJQPK68UPPaXsYBL98BptE7vUcYrIrVseDjcjXOV5+6oAAAAAAAAAAAAAAAABxLdLQBMS0AAAAAAAsubsgAAAAAAAAADAAAAAVNMVAAAAAAAlA8rrxQ89pexgEv3wGm0Tu9RxisitWx4ONyNc5Xn7qgAAAAAAAAAAAAAAAAA5Gb5AAmJaAAAAAACy6UiAAAAAAAAAAMAAAABU0xUAAAAAACUDyuvFDz2l7GAS/fAabRO71HGKyK1bHg43I1zlefuqAAAAAAAAAAAAAAAAALfSBMAHoSAAAAAAALLpSMAAAAAAAAAAwAAAAFTTFQAAAAAAJQPK68UPPaXsYBL98BptE7vUcYrIrVseDjcjXOV5+6oAAAAAAAAAAAAAAAABz7NkwBMS0AAAAAAAsulJAAAAAAAAAADAAAAAVNMVAAAAAAAlA8rrxQ89pexgEv3wGm0Tu9RxisitWx4ONyNc5Xn7qgAAAAAAAAAAAAAAAAO16AnAJiWgAAAAAACy66eAAAAAAAAAAMAAAABU0xUAAAAAACUDyuvFDz2l7GAS/fAabRO71HGKyK1bHg43I1zlefuqAAAAAAAAAAAAAAAAAd3OxUATEtAAAAAAALLrp8AAAAAAAAAAwAAAAFTTFQAAAAAAJQPK68UPPaXsYBL98BptE7vUcYrIrVseDjcjXOV5+6oAAAAAAAAAAAAAAAAAB4h9AABMS0AAAAAAsuuoAAAAAAAAAADAAAAAVNMVAAAAAAAlA8rrxQ89pexgEv3wGm0Tu9RxisitWx4ONyNc5Xn7qgAAAAAAAAAAAAAAAAPKCsDAJiWgAAAAAACy66hAAAAAAAAAAMAAAAAAAAAAVNMVAAAAAAAlA8rrxQ89pexgEv3wGm0Tu9RxisitWx4ONyNc5Xn7qgAAAAAAAAAAAADfDsATEtAAAAAAALLlB4AAAAAAAAAAwAAAAAAAAABU0xUAAAAAACUDyuvFDz2l7GAS/fAabRO71HGKyK1bHg43I1zlefuqAAAAAAAAAAAAAOlMwBMS0AAAAAAAsuUGwAAAAAAAAADAAAAAAAAAAFTTFQAAAAAAJQPK68UPPaXsYBL98BptE7vUcYrIrVseDjcjXOV5+6oAAAAAAAAAAAAB0//AJiWgAAAAAACy5QcAAAAAAAAAAMAAAAAAAAAAVNMVAAAAAAAlA8rrxQ89pexgEv3wGm0Tu9RxisitWx4ONyNc5Xn7qgAAAAAAAAAAAAHWzsAmJaAAAAAAALLntkAAAAAAAAAAwAAAAAAAAABU0xUAAAAAACUDyuvFDz2l7GAS/fAabRO71HGKyK1bHg43I1zlefuqAAAAAAAAAAAAAA7BwAExLQAAAAAAsuh8wAAAAAAAAADAAAAAAAAAAFTTFQAAAAAAJQPK68UPPaXsYBL98BptE7vUcYrIrVseDjcjXOV5+6oAAAAAAAAAAAAB2aJAJiWgAAAAAACy6H0AAAAAAAAAAMAAAAAAAAAAVNMVAAAAAAAlA8rrxQ89pexgEv3wGm0Tu9RxisitWx4ONyNc5Xn7qgAAAAAAAAAAAAAAw8AAD6AAAAAAALLofUAAAAAAAAAAwAAAAAAAAABU0xUAAAAAACUDyuvFDz2l7GAS/fAabRO71HGKyK1bHg43I1zlefuqAAAAAAAAAAAAAeOpQCYloAAAAAAAsuh9gAAAAAAAAADAAAAAAAAAAFTTFQAAAAAAJQPK68UPPaXsYBL98BptE7vUcYrIrVseDjcjXOV5+6oAAAAAAAAAAAAA8o5AExLQAAAAAACy6hGAAAAAAAAAAMAAAAAAAAAAVNMVAAAAAAAlA8rrxQ89pexgEv3wGm0Tu9RxisitWx4ONyNc5Xn7qgAAAAAAAAAAAAHmkMAmJaAAAAAAALLqEcAAAAAAAAAAwAAAAAAAAABU0xUAAAAAACUDyuvFDz2l7GAS/fAabRO71HGKyK1bHg43I1zlefuqAAAAAAAAAAAAAD0AwATEtAAAAAAAsuoSAAAAAAAAAADAAAAAAAAAAFTTFQAAAAAAJQPK68UPPaXsYBL98BptE7vUcYrIrVseDjcjXOV5+6oAAAAAAAAAAAAB6vRAJiWgAAAAAACy6hJAAAAAAAAAAMAAAAAAAAAAVNMVAAAAAAAlA8rrxQ89pexgEv3wGm0Tu9RxisitWx4ONyNc5Xn7qgAAAAAAAAAAAAD280ATEtAAAAAAALLqEoAAAAAAAAAAwAAAAAAAAABU0xUAAAAAACUDyuvFDz2l7GAS/fAabRO71HGKyK1bHg43I1zlefuqAAAAAAAAAAAAAGP4QAehIAAAAAAAsuoSwAAAAAAAAADAAAAAAAAAAFTTFQAAAAAAJQPK68UPPaXsYBL98BptE7vUcYrIrVseDjcjXOV5+6oAAAAAAAAAAAAB9VjAJiWgAAAAAACy6hMAAAAAAAAAAMAAAAAAAAAAVNMVAAAAAAAlA8rrxQ89pexgEv3wGm0Tu9RxisitWx4ONyNc5Xn7qgAAAAAAAAAAAAH858AmJaAAAAAAALLqE0AAAAAAAAAAwAAAAAAAAABU0xUAAAAAACUDyuvFDz2l7GAS/fAabRO71HGKyK1bHg43I1zlefuqAAAAAAAAAAAAAf5uQCYloAAAAAAAsuoTgAAAAAAAAADAAAAAAAAAAFTTFQAAAAAAJQPK68UPPaXsYBL98BptE7vUcYrIrVseDjcjXOV5+6oAAAAAAAAAAAAAP/7ABMS0AAAAAACy6hPAAAAAAAAAAFmhpEOAAAAQDxFLGrtAeyuRXaMNWp9tG9IxhrGv7yYbi9yM8lKayXCkY1Y92vB61AZVkZCh+SXrm/m8VOdP4fBalMBhqzoigk=
2018/11/21 05:54:03 submitting tx XDR to network (async)
2018/11/21 05:55:03 (async) error: tx failed for unknown reason, error message: error decoding horizon.Problem: EOF
2018/11/21 05:55:03 ----------------------------------------------------------------------------------------------------

Expected behavior

Horizon calls should work.

Frequency

As of now it's always. The bot had been running for about 13 hours successfully before the errors started.

Steps To Reproduce

On my end just try to start the bot, it will only get as far as trying to validate trustlines and then return error decoding horizon.Problem: EOF Update: when I retried one more time before submitting this issue it got past validating trustlines and failed at loading offers.

Possible Solution

I think this is probably a server side issue but I wanted to report it because it's continual now and I wouldn't expect horizon to be down for 2+ hours. I don't know what would be wrong client side.

Your Environment

version: bal_reserve:v1.1.1-32-g4b6b46e9-dirty
git branch: bal_reserve
git hash: 4b6b46e971bca7f0269aa951eee27a4e88df8f0b-dirty
build date: 20181120T001603Z
GOOS: linux
GOARCH: amd64

Context

The bot does not function in this state.

Desired Behavior

I want to be able to save log entries in between runs of kelp automatically

Impact

The desired behavior will allow me to retrospectively look back at the bot's actions without having to manually rotate log files

Feature Suggestion

We can achieve the desired behavior by outputting log lines directly to a file rather than to stdout that has a date suffix in the name

References

This is a common feature in almost all applications

Describe the bug

build.sh returns "need to invoke from the root 'kelp' directory" despite being in the kelp directory.

If I remove the code that triggers the error and run the script in go\src\github.com\lightyeario\kelp the build succeeds but generates a \bin directory as go\src\github.com\lightyeario\kelp\bin and places the executable there without the .exe extension. Manually adding .exe to the file creates a functional application.

Expected behavior

build.sh should build the executable and place it in go/bin

Frequency

Always occurs until edit is made

Steps To Reproduce

Here are the steps to reproduce the issue:

1. Follow the steps for compiling from source as documented in README.md, through step 4, successfully

2. Run /scripts/build.sh which will return the "need to invoke from the root 'kelp' directory" error

3. Edit build.sh and remove this code :

   if [[ ($# -gt 1 || 'pwd | rev | cut -d'/' -f1 | rev' != "kelp") ]] then echo "need to invoke from the root 'kelp' directory" exit 1 fi

4. Move build.sh to go\src\github.com\lightyeario\kelp

5. Run build.sh

6. You should have a go\src\github.com\lightyeario\kelp\bin directory now, containing an untyped "kelp" file

7. Add .exe to the file

8. It should now be a functioning application

Your Environment

// Run kelp version to get the version and build information and paste it here
version: v1.0.0-rc3-1-g09ae14cf-dirty
git hash: 09ae14c-dirty
build date: 20181002T003349Z
GOOS: windows
GOARCH: amd64

Context

Until I found the workaround I couldn't build from source code, preventing me from trying to contribute to the project. So I'm fine now, but it would be good to fix for other users.

Desired Behavior

I would like to be able to customize more of Kelp's logic than what is currently possible. Currently custom strategy logic is confined mostly to the level providers. However, strategic logic also resides in parts of sellSideStrategy, namely UpdateWithOps and modifySellLevel. Having that logic in a static part of the code limits customization possibilities.

Impact

This would enable users to create more complex customized strategies without messing with sellSideStrategy and changing things for all strategies. I think it also makes sense to have functions that contain level-setting logic be part of the strategy code in general.

Feature Suggestion

We could do this by converting UpdateWithOps and modifySellLevel to work the way level providers do. They would be constructed as part of strategy generation

Additional context

The level providers do allow quite a bit of customization, including the idea I was thinking through when I thought of this, #52. However, the proposed change would enable much more complex logic than that issue.

Specification

We would have the functions in strategy specific files, similar to the files that define level providers. The current static version would be kept for use by strategies that don't need to customize these functions.

The part of the strategy factory methods that generates the sellSideStrategy would change to be something like this:

sellSideStrategy := makeSellSideStrategy(
		sdex,
		assetBase,
		assetQuote,
		makeBalancedLevelProvider(
			config.Spread,
			false,
			config.MinAmountSpread,
			config.MaxAmountSpread,
			config.MaxLevels,
			config.LevelDensity,
			config.EnsureFirstNLevels,
			config.MinAmountCarryoverSpread,
			config.MaxAmountCarryoverSpread,
			config.CarryoverInclusionProbability,
			config.VirtualBalanceBase,
			config.VirtualBalanceQuote),
                makeBalancedUpdater(
                        ParamOne,
                        ParamTwo,)
		config.PriceTolerance,
		config.AmountTolerance,
		false,
	)

A new type would be needed in api/level:

type Updater interface {
	UpdateWithOps(offers []horizon.Offer) (ops []build.TransactionMutator, newTopOffer *model.Number, e error) ,
	modifySellLevel(offers []horizon.Offer, index int, targetPrice model.Number, targetAmount model.Number) (*model.Number, bool, *build.ManageOfferBuilder, error)
}

A additional parameter would be added to sellSideStrategy and makeSellSideStrategy

type sellSideStrategy struct {
	sdex                *SDEX
	assetBase           *horizon.Asset
	assetQuote          *horizon.Asset
	levelsProvider      api.LevelProvider
        updater              api.Updater
	priceTolerance      float64
	amountTolerance     float64
	divideAmountByPrice bool
	action              string

	// uninitialized
	currentLevels []api.Level // levels for current iteration
	maxAssetBase  float64
	maxAssetQuote float64
}

// ensure it implements SideStrategy
var _ api.SideStrategy = &sellSideStrategy{}

// makeSellSideStrategy is a factory method for sellSideStrategy
func makeSellSideStrategy(
	sdex *SDEX,
	assetBase *horizon.Asset,
	assetQuote *horizon.Asset,
	levelsProvider api.LevelProvider,
        updater api.Updater,
	priceTolerance float64,
	amountTolerance float64,
	divideAmountByPrice bool,
) api.SideStrategy {
	action := actionSell
	if divideAmountByPrice {
		action = actionBuy
	}
	return &sellSideStrategy{
		sdex:                sdex,
		assetBase:           assetBase,
		assetQuote:          assetQuote,
		levelsProvider:      levelsProvider,
		priceTolerance:      priceTolerance,
		amountTolerance:     amountTolerance,
		divideAmountByPrice: divideAmountByPrice,
		action:              action,
	}
}

I may well be missing somewhere else a change would be needed. This is obviously a big change, so I'm not going to code it up for a pull request unless approved.

related to #40

Describe the bug

The bot is unable to submit transactions if it has hit capacity limits for non-XLM assets. This requires at least 2 levels in the strategy.

Expected behavior

I think the correct behavior in this situation is for the bot to handle this smoothly.

Frequency

The frequency is: Sometimes (in the above scenario)

Steps To Reproduce

Here are the steps to reproduce the issue:

1. The bot should be configured to place at least 2 levels.
2. Use a price feed such as kraken so there is some variability in the asset price.
3. Configure the bot with the buysell strategy for an amount that will more than max out the non-XLM asset it is selling after it successfully places the full first level.
4. Start the bot.
5. Very soon the bot will have op_underfunded errors in the transaction result for the first operation.

Possible Solution

The native XLM asset is configured to provide a operational buffer of 20 XLM which prevents this issue from being hit with the XLM asset. A similar buffer can be added for non-native assets.

Your Environment

  version: master:v1.1.1-28-gdb5615eb
  git branch: master
  git hash: db5615ebcf4be719c9e71a2c8778ca30eae2d289
  build date: 20181126T221437Z
  GOOS: linux
  GOARCH: amd64

Context

I am unable to adjust the bot's amounts as desired or run the bot reliably because of this bug.

Attachments

2018/11/26 14:25:42 Starting the trader bot...
2018/11/26 14:25:42 ----------------------------------------------------------------------------------------------------
2018/11/26 14:25:42  (base) assetA=native, maxA=15653.0916464, trustA=math.MaxFloat64
2018/11/26 14:25:42 (quote) assetB=TEST:GCFFRPDGJHMTMP5WAVEUSDIYNWCDUZTRE2QMLYVVPAYYFKZPMF4AMNTG, maxB=740.9049244, trustB=922337203685.4775391
2018/11/26 14:25:42 liabilities after resetting
2018/11/26 14:25:42 asset=base  , balance=15653.0916464, trust=math.MaxFloat64, minAccountBal=23.5000000, buyingLiabilities=0.0000000, sellingLiabilities=0.0000000
2018/11/26 14:25:42 asset=quote , balance=740.9049244, trust=922337203685.4775391, minAccountBal=0.0000000, buyingLiabilities=0.0000000, sellingLiabilities=0.0000000
2018/11/26 14:25:43 price from exchange feed (ccxt-binance/XLM/USDT): bidPrice=0.1441200, askPrice=0.1448500, centerPrice=0.1444850
2018/11/26 14:25:43 feedPair prices: feedA=1.0000000, feedB=0.1444850; centerPrice=6.9211337
2018/11/26 14:25:44 price from exchange feed (ccxt-binance/XLM/USDT): bidPrice=0.1441400, askPrice=0.1448500, centerPrice=0.1444950
2018/11/26 14:25:44 feedPair prices: feedA=0.1444950, feedB=1.0000000; centerPrice=0.1444950
2018/11/26 14:25:44 offer | buy  | level=1 | curPriceQuote=0.1317045 | curAmtBase=0.1000003 | pruning=false
2018/11/26 14:25:44 offer | buy  | level=2 | curPriceQuote=0.1207292 | curAmtBase=6130.6714067 | pruning=false
2018/11/26 14:25:44 offer | sell | level=1 | curPriceQuote=0.1594175 | curAmtBase=0.0999997 | pruning=false
2018/11/26 14:25:44 offer | sell | level=2 | curPriceQuote=0.1739100 | curAmtBase=10000.0000000 | pruning=false
2018/11/26 14:25:44 created 0 operations to prune excess offers
2018/11/26 14:25:44 liabilities after resetting
2018/11/26 14:25:44 asset=base  , balance=15653.0916464, trust=math.MaxFloat64, minAccountBal=23.5000000, buyingLiabilities=0.0000000, sellingLiabilities=0.0000000
2018/11/26 14:25:44 asset=quote , balance=740.9049244, trust=922337203685.4775391, minAccountBal=0.0000000, buyingLiabilities=0.0000000, sellingLiabilities=0.0000000
2018/11/26 14:25:44 buy  | modify | level=1 | targetPriceQuote=0.1313500 | targetAmtBase=0.1000000 | curPriceQuote=0.1317045 | lowPriceQuote=0.1313500 | highPriceQuote=0.1313500 | curAmtBase=0.1000003 | minAmtBase=0.0947443 | maxAmtBase=0.1047173
2018/11/26 14:25:44 buy  | modify | level=2 | targetPriceQuote=0.1204042 | targetAmtBase=10000.0000002 | curPriceQuote=0.1207292 | lowPriceQuote=0.1204042 | highPriceQuote=0.1204042 | curAmtBase=6130.6714067 | minAmtBase=9474.4262635 | maxAmtBase=10471.7342912
2018/11/26 14:25:44 we will oversell the asset 'TEST:GCFFRPDGJHMTMP5WAVEUSDIYNWCDUZTRE2QMLYVVPAYYFKZPMF4AMNTG', amountSelling = 1204.0416693, bal = 740.9049244, minAccountBal = 0.0000000, liabilities.Selling = 0.0131350
2018/11/26 14:25:44 computed remainder amount, constrained by selling capacity, returning sellingAmount=740.8917894, buyingAmount=6153.3733284
2018/11/26 14:25:44 buy  | modify | level=2 | targetPriceQuote=0.1204042 | targetAmtBase=6153.3733284 | curPriceQuote=0.1207292 | lowPriceQuote=0.1204042 | highPriceQuote=0.1204042 | curAmtBase=6130.6714067 | minAmtBase=9474.4262635 | maxAmtBase=10471.7342912
2018/11/26 14:25:44 sell | modify | level=1 | targetPriceQuote=0.1589445 | targetAmtBase=0.1000000 | curPriceQuote=0.1594175 | lowPriceQuote=0.1589445 | highPriceQuote=0.1589445 | curAmtBase=0.0999997 | minAmtBase=0.0950000 | maxAmtBase=0.1050000
2018/11/26 14:25:44 sell | modify | level=2 | targetPriceQuote=0.1733940 | targetAmtBase=10000.0000000 | curPriceQuote=0.1739100 | lowPriceQuote=0.1733940 | highPriceQuote=0.1733940 | curAmtBase=10000.0000000 | minAmtBase=9500.0000000 | maxAmtBase=10500.0000000
2018/11/26 14:25:44 liabilities at the end of a call to UpdateWithOps
2018/11/26 14:25:44 asset=base  , balance=15653.0916464, trust=math.MaxFloat64, minAccountBal=23.5000000, buyingLiabilities=6153.4733284, sellingLiabilities=10000.1000000
2018/11/26 14:25:44 asset=quote , balance=740.9049244, trust=922337203685.4775391, minAccountBal=0.0000000, buyingLiabilities=1733.9558944, sellingLiabilities=740.9049244
2018/11/26 14:25:44 created 4 operations to update existing offers
2018/11/26 14:25:44 reloading sequence number
2018/11/26 14:25:44 tx XDR: AAAAAIpYvGZJ2TY/tgVJSQ0YbYQ6ZnEmoMXitXgxgqsvYXgGAAABkAAAMKUAAAbsAAAAAAAAAAAAAAAEAAAAAQAAAAC+B05WMqwKNPp1zsO8wisjLSFafnHK96h/lpSKbpq9fgAAAAMAAAABVEVTVAAAAACKWLxmSdk2P7YFSUkNGG2EOmZxJqDF4rV4MYKrL2F4BgAAAAAAAAABuZsdhgE80wkAJiWgAAAAAAACB6oAAAABAAAAAL4HTlYyrAo0+nXOw7zCKyMtIVp+ccr3qH+WlIpumr1+AAAAAwAAAAFURVNUAAAAAIpYvGZJ2TY/tgVJSQ0YbYQ6ZnEmoMXitXgxgqsvYXgGAAAAAAAAAAAAAgEWAHQrPwAPQkAAAAAAAAIHyQAAAAEAAAAAvgdOVjKsCjT6dc7DvMIrIy0hWn5xyveof5aUim6avX4AAAADAAAAAAAAAAFURVNUAAAAAIpYvGZJ2TY/tgVJSQ0YbYQ6ZnEmoMXitXgxgqsvYXgGAAAAAAAPQkAABNnBAB6EgAAAAAAAAgeIAAAAAQAAAAC+B05WMqwKNPp1zsO8wisjLSFafnHK96h/lpSKbpq9fgAAAAMAAAAAAAAAAVRFU1QAAAAAili8ZknZNj+2BUlJDRhthDpmcSagxeK1eDGCqy9heAYAAAAXSHboAAABUqkAB6EgAAAAAAACA/4AAAAAAAAAAi9heAYAAABABEgxWWvPy/ugeRuxsDUldFMdwv+S2/YSPIef4lMtxzpjIp1k6inBY3nbn9JuwQ+dVBUOGOvVOPsV9ERAg83wBW6avX4AAABA6CHI0nICXxVYfiVzNHid7MbbF2ApejvtyWqVQLROGgfuCbDh710RyG9lFhHQmNW5nLWLId29ytFJJS3M0DGmBA==
2018/11/26 14:25:44 submitting tx XDR to network (async)
2018/11/26 14:25:50 (async) error: result code details: tx code = tx_failed , opcodes = [op_underfunded op_success op_success op_success]

Desired Behavior

Travis test run should fail of the build is unable to start up and trade

Impact

The desired behavior will allow me to confidently merge pull requests without having to sync the code and test locally

Feature Suggestion

We can achieve the desired behavior by having a simple integration test that generates the required accounts, assets, and trustlines, funding, and runs a simple trading strategy.

This should be run as a go test along with all the other existing tests.

Testing against the test network is ok for now (vs. spinning up a new private testnet)

This can run against the buysell strategy

Desired Behavior

I want to be able to have the bot use a greater % of my balance per level when making shallower orderbooks.

Impact

This would let me actually use more of my asset total in a reasonable spread. Right now 25 levels at spread 0.002 goes out 4.5% in each direction and only uses only ~4.7% of my balance. (which I realize now is intended behavior, see below comments)

Feature Suggestion

Add a parameter to the balanced config file to manipulate the amount-per-level.

Specification

This spec won't work, see comments below. Leaving it here to show what I was thinking.

Assuming I'm reading the code correctly, the parameter would affect the calculations in the getLevel function in balancedLevelProvider.go, lines 193-196, replacing the static 2 multiplier with a passed value.

	targetAmount := (2 * maxAssetBase * p.spread) / (4 + p.spread)
	if p.useMaxQuoteInTargetAmountCalc {
		targetAmount = (2 * maxAssetQuote * p.spread) / (4 + p.spread)
	}

Describe the bug

The following conditions have to be met in order for the bot to get an op_underfunded error:

• current levels needs to be > 1
• bot is currently hitting capacity limits
• bot tries to increase order size of first level but reduce total size of orders such that it will not be hitting capacity limits anymore

Expected behavior

I think the correct behavior in this situation is for the bot to handle these cases correctly without having to manually delete all the offers as a workaround

Frequency

The frequency is: Always in the described situation

Steps To Reproduce

Here are the steps to reproduce the issue (see attachments in section below):

1. Set up the bot to use two levels with the first level being small and the second level consuming the remaining balance so it hits capacity limits
2. once the orderbook is updated to this setting, stop the bot and update the levels such that the first level is bigger and the second level is less then what remains from the balance such that the bot will no longer hit capacity limits.
3. Start the bot, you will see the op_underfunded error because the bot tries to increase the first level before trying to reduce the second level which leaves it in a state where it cannot fund the first operation.

Possible Solution

This bug can be fixed by reordering operations when we modify a level that will reduce the size of an order even if we will not hit capacity limits.

Your Environment

  version: master:v1.2.0-2-g5b6224b7
  git branch: master
  git hash: 5b6224b7b47fd3ed9f22ad3f127dce787691324d
  build date: 20181202T000856Z
  GOOS: linux
  GOARCH: amd64

Describe the bug

The bot crashes on startup when trying to print the config details if you don't include the SOURCE_SECRET_SEED in the trader config file.

I think this behavior is incorrect because the account setup walkthrough and sample config file says that the SOURCE_SECRET_SEED is optional.

Here is the log of the error:

2018/08/24 14:41:14 Starting Kelp Trader: v1.0.0-rc1 [337f478b0e7a1dc235aef31788754bc1ab11b6a1]
2018/08/24 14:41:14 configs:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1358134]

goroutine 1 [running]:
github.com/lightyeario/kelp/support/utils.SecretKey2PublicKey(0x140ddc0, 0xc42025ea50, 0x98, 0x140ddc0)
	~/dev/go/src/github.com/lightyeario/kelp/support/utils/configs.go:62 +0xd4
github.com/lightyeario/kelp/support/utils.StructString(0x14a9740, 0xc42025ea50, 0xc4201e5450, 0xc42025ea50, 0xc4201e5510)
	~/dev/go/src/github.com/lightyeario/kelp/support/utils/configs.go:40 +0x2b2
github.com/lightyeario/kelp/trader.BotConfig.String(0x0, 0x0, 0xc4201bd040, 0x38, 0xc4201c0f60, 0x3, 0x0, 0x0, 0xc4201c0f80, 0x6, ...)
	~/dev/go/src/github.com/lightyeario/kelp/trader/config.go:32 +0x180
github.com/lightyeario/kelp/support/utils.CheckConfigError(0x151d0c0, 0xc42025e960, 0x0, 0x0, 0x7fff5fbff52c, 0x11)
	~/dev/go/src/github.com/lightyeario/kelp/support/utils/configs.go:20 +0x198
github.com/lightyeario/kelp/cmd.init.4.func1(0x175f680, 0xc420081440, 0x0, 0x6)
	~/dev/go/src/github.com/lightyeario/kelp/cmd/trade.go:65 +0x287
github.com/lightyeario/kelp/vendor/github.com/spf13/cobra.(*Command).execute(0x175f680, 0xc4200813e0, 0x6, 0x6, 0x175f680, 0xc4200813e0)
	~/dev/go/src/github.com/lightyeario/kelp/vendor/github.com/spf13/cobra/command.go:702 +0x2c6
github.com/lightyeario/kelp/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x175fce0, 0x1005d9c, 0xc42002c118, 0x0)
	~/dev/go/src/github.com/lightyeario/kelp/vendor/github.com/spf13/cobra/command.go:783 +0x2e4
github.com/lightyeario/kelp/vendor/github.com/spf13/cobra.(*Command).Execute(0x175fce0, 0x0, 0x13becf2)
	~/dev/go/src/github.com/lightyeario/kelp/vendor/github.com/spf13/cobra/command.go:736 +0x2b
main.main()
	~/dev/go/src/github.com/lightyeario/kelp/main.go:10 +0x2d

Expected behavior

I think the correct behavior in this situation is for the bot to not crash if SOURCE_SECRET_SEED is missing or empty. The value printed out to the display can always be an empty value in this case.

Frequency

The frequency is: Always

Steps To Reproduce

Here are the steps to reproduce the issue (see attachments in section below):

1. Set up your configuration file with the SOURCE_SECRET_SEED field to be empty
2. Start the bot using this command:
   ./kelp trade -c sample_trader.cfg -s buysell -f sample_buysell.cfg --sim
3. The bot will crash with the error mentioned above.

Possible Solution

This bug can be fixed by allowing an empty value for the secret key when printing the config.

Your Environment

version: v1.0.0-rc1
git hash: 337f478b0e7a1dc235aef31788754bc1ab11b6a1
build date: 20180813T231816Z
GOOS: linux
GOARCH: amd64

Context

I am unable to use the same account as the trader account and the source account because of this bug. As a workaround, I'm currently copy-pasting the value in the TRADER_SECRET_SEED for the SOURCE_SECRET_SEED, although I know that's not the intended usage.

Attachments

No relevant attachments

Describe the bug

If the bot's strategy logic results in no levels being generated for the sell side, buy levels are not checked for refresh. This does not happen in the opposite direction; if there are no buy levels, sell levels are still checked. No error is thrown.

Expected behavior

The bot should evaluate existing levels and generate new levels as normal regardless of whether the bot is placing orders on the other side.

Frequency

Always if the conditions are met.

Steps To Reproduce

1. Set up a strategy that sometimes validly creates no levels on one side of the orderbook. I'm using my strategy submitted in PR #39. In this case the MAINTAIN_BALANCE_PERCENT parameter instructs the strategy to place no orders on one side when that side's balance is below a certain threshold.
2. Set the parameter such that it places no sell orders, e.g. MAINTAIN_BALANCE_PERCENT higher than the sold asset balance ratio but lower than the buy asset's ratio.
3. Set PRICE_TOLERANCE low so the bot should update orders frequently.
4. Run the bot. It will successfully place buy orders on the first cycle, presumably because it is expecting no existing levels.
5. Wait for the price to change towards the sell side enough to trigger PRICE_TOLERANCE.
6. The buy orders will sit as they are regardless of increase in price.

Possible Solution

I think this can be fixed by removing the break on line 130 of sellSideStrategy. This part:

if hitCapacityLimit {
			if isModify {
				delOp := s.sdex.DeleteOffer(offers[i])
				log.Printf("deleting offer because we previously hit the capacity limit, offerId=%d\n", offers[i].ID)
				deleteOps = append(deleteOps, delOp)
				continue
			} else {
				// we can break because we would never see a modify operation happen after a non-modify operation
				break
			}
		}

Using a strategy with valid asymmetrical orders I think you can see a modify operation after a non-modify operation. In this case, I think the buy-side levels aren't being checked because the loop is breaking before it gets to them. Which would line up with this being a one-sided problem; if the sell levels are checked first then a break triggered by no buy levels would be after the sell levels are already done.

I'm now trying a version with the break replaced with another continue to see if the problem occurs in that case; I'll update if I have clear evidence either way.

Your Environment

version: v1.1.1-9-g59a9340e-dirty
git hash: 59a9340-dirty
build date: 20181030T233844Z
GOOS: linux
GOARCH: amd64

Context

I am unable to run strategies with asymmetrical logic. This could also possibly affect things like a maker-only mode where some levels could be validly skipped.

This issue has been partially completed in #28, which leaves out the expansion of the CCXT integration

Desired Behavior

I want to connect my strategies to more than just the Kraken exchange.

Impact

The desired behavior will allow me to get better data and will allow me to offset my assets on more exchanges.

Feature Suggestion

We can achieve the desired behavior by integrating the CCXT library (REST API) into the bot's framework.

We will need to deploy the REST wrapper as a micro-service and create an adapter for this REST API that will give us access to the many exchanges supported by CCXT. The adapter will allow us to maintain the exchange interface that we currently have for the bot, thus allowing it to work seamlessly with our existing strategies.

References

This feature exists in many other cryptocurrency repos.
Here's a link to the CCXT library along with a REST wrapper around it for use in Golang:
CCXT GitHub: https://github.com/ccxt/ccxt
REST wrapper around CCXT: https://github.com/franz-see/ccxt-rest

Additional context

Some alternatives that would achieve the same result are:

• Invoke python/node code directly using IPC or queues
• manually integrate the libraries for each individual exchange

The feature suggested above was the better than these alternatives because by creating a microservice for CCXT, we can decouple the exchange integration from the bot, easily stay up-to-date with any API changes for all these exchanges, and deploy 100+ crypto exchanges with a simple adapter exchange implementation.

Describe the bug

Currently (last commit is e38542c) the master codebase fails to compile with these errors:

# github.com/interstellar/kelp/plugins
./fillTracker.go:72:36: too many arguments in call to f.threadTracker.TriggerGoroutine
	have (func([]interface {}), []interface {})
	want (func())
./sdex.go:411:38: too many arguments in call to sdex.threadTracker.TriggerGoroutine
	have (func([]interface {}), nil)
	want (func())
./sdex.go:474:37: too many arguments in call to sdex.threadTracker.TriggerGoroutine
	have (func([]interface {}), nil)
	want (func())

This is without any changes on my part.

Expected behavior

The master branch should compile

Frequency

Currently, always.

Steps To Reproduce

Pull the current master code to your local branch and run ./scripts/build.sh

Possible Solution

Removing the extra arguments would presumably clear the error check, but I don't know if removing them would cause other problems.

Your Environment

version: master:v1.1.1-54-ge38542cd
git branch: master
git hash: e38542cd0652f00bad6f17f3df2db7568553d673
build date: 20181204T025348Z
GOOS: linux
GOARCH: amd64

Context

I can't test modifications if the base code doesn't compile.

Describe the bug

Sometimes the bot will not replace orders on the side of the orderbook where the price has moved towards but will re-place at least one order from the other side to cross the original orders that were not re-placed. This will cause the full update transaction to fail with an op_cross_self error.

This only happens when we have PRICE_TOLERANCE set to be non-zero in the configs. See the Frequency section for the specific conditions under which this occurs.

This is incorrect because the PRICE_TOLERANCE should not cause the full update transaction to fail.

Expected behavior

I think the correct behavior in this situation is for the bot to re-place the order that needs updating with the "old" price, since it was triggered only by failing the AMOUNT_TOLERANCE check and not by the PRICE_TOLERANCE check.

Frequency

The frequency is: Sometimes.

Currently this is my best guess on when this happens. All the following conditions need to be met at the same time in order for the bug to be triggered:

• PRICE_TOLERANCE config setting is >> 0 (example: 0.01)
• the center price has moved (enough so that the spread would be crossed if we were to re-place orders)
• None of the orders on the side where the price has moved should trigger re-placement.
• the first order at the tip of the orderbook (on the side away from where the price has moved) should have less then it's full amount of orders placed so it triggers an order replacement because it fails the AMOUNT_TOLERANCE check, but passes the PRICE_TOLERANCE check.

This causes the one order to be placed crossing over the existing order (since the bot decided to not re-place the orders on the side where the price moved to), causing the op_cross_self error.

Steps To Reproduce

Here are the steps to reproduce the issue (see attachments in section below):

1. Set up your configuration file with these parameters: PRICE_TOLERANCE = 0.05
2. Start the bot using the regular command of the buysell strategy.
3. Update the price source in your price feed to be within the limit of the PRICE_TOLERANCE but enough to cross the spread.
4. Consume enough of your order on the side away from where the price moved, enough to trigger the AMOUNT_TOLERANCE check.
5. Wait until the bot picks up the change and watch the logs, you will notice an op_cross_self error and the bot will continue to move to the next update cycle.
6. This error will continue every update cycle until the priceFeed changes again to prevent this error.

Possible Solution

This bug can be fixed by the bot using the old price for orders that get triggered by only the AMOUNT_TOLERANCE.

Your Environment

version: v1.0.0-rc1
git hash: 337f478b0e7a1dc235aef31788754bc1ab11b6a1
build date: 20180813T231816Z
GOOS: linux
GOARCH: amd64

Context

I am unable to use the bot reliably because of this bug.

Attachments

No relevant attachments.

Desired Behavior

I want to be able to see the selected strategy in the log output of the bot

Impact

The desired behavior will allow me to better analyze logs in retrospect

Feature Suggestion

We can achieve the desired behavior by printing out this value before the printed configs

Describe the bug

The bot crashes when it cannot handle the response from CCXT. This should be handled more gracefully

Expected behavior

I think the correct behavior in this situation is for the bot to check the type of the returned value and its existence before forcing a cast

Frequency

The frequency is: Sometimes

Steps To Reproduce

Here are the steps to reproduce the issue (see attachments in section below):

1. Set up your configuration to use an exchange from the CCXT integration
2. Set up the CCXT endpoint in such a way that it does not return a valid value for the get ticker information (or maybe even mock it out in the code that parses it)
3. run the bot, it will crash.

Possible Solution

This bug can be fixed by checking the returned value before forcing a type assertion

Your Environment

version: v1.1.0
git hash: 1a62077
build date: 20181022T210554Z
GOOS: linux
GOARCH: amd64

Context

I am unable to run the bot reliably because of this bug.

Attachments

panic: interface conversion: interface {} is nil, not float64

goroutine 1 [running]:
github.com/lightyeario/kelp/plugins.ccxtExchange.GetTickerPrice(0xc42002b070, 0x8b51f5, 0x1, 0xc42041a080, 0x7, 0xc42054e320, 0x1, 0x1, 0xa7, 0x280, ...)
        ~/dev/go/src/github.com/lightyeario/kelp/plugins/ccxtExchange.go:53 +0x693
github.com/lightyeario/kelp/plugins.(*exchangeFeed).GetPrice(0xc4202664b0, 0x280, 0xa7, 0x1aa3b6c4c6f1)
        ~/dev/go/src/github.com/lightyeario/kelp/plugins/exchangeFeed.go:32 +0x67

buysell config file

When changing the buysell configuration file. to read krakens xlm/usd orders to xlm/euro
The bot does not understand the xlm euro pair, and throws an error.

I think the correct behavior in this situation is for the bot to ...

Steps To Reproduce

Open config file and change what is read from kraken from xxlm/zusd to xxlm/zeur
https://support.kraken.com/hc/en-us/articles/360001185506-Asset-Codes

Desired Behavior

I want to be able to debug IEIF/CAP-3 issues just by looking at log files, which should contain asset balance data when logging out liability data.

Impact

The desired behavior will allow me to do so since the data needed from horizon would be included in the log.

Feature Suggestion

We can achieve the desired behavior by fetching the horizon balance anytime we log liability data and including that in the log line.

We should also consider including the liability information that we get from horizon so we can compare in case there are any errors in our liability calculations.

Desired Behavior

I want to be able to see the target price and target amount when modifying an existing sell offer

Impact

The desired behavior will allow me to see what the new price and amount will be for a given update cycle.

Feature Suggestion

// A succinct description of how you want to achieve the desired behavior.
// If you have looked at the code and know exactly what code changes are needed then please consider submitting a pull request instead.
// If this feature does not exist anywhere else then please restrict any details to the Specification section below.

We can achieve the desired behavior by updating this log line in sellSideStrategy.go.

Desired Behavior

I want to be able to use the offset trades functionality of the mirror strategy knowing that it will only place orders that can be covered on the backing exchange

Impact

The desired behavior will allow me to confidently run the mirror strategy by offsetting my trades

Feature Suggestion

We can achieve the desired behavior by using the balance on the backing exchange to limit the offers we place on the primary exchange.

References

This feature does not exist anywhere else that I'm aware of.

We are currently testing kelp with small amount of funds in the live market for our project. This is an amazing tool and thank you interstellar team for creating this. It is working great with small amount of funds and we are planning to increase it but I have a question before we do that.
Our purpose is to create liquidity for the token.

Question
Suppose we are using kelp with balanced config and fund the bot with equal value of both asset. The bots start working and based on config parameters, it will create buy/sell offers. Suppose someone sends more funds of either asset, now in that the value of both asset will not be equal in that bot account. It might hamper the functionality of the bot and an attacker can use this to manipulate the price.
Is there any funntion in Kelp bot that if unwanted funds(of either of the assets) are sent to bot account, it will not hamper the functionality of Kelp ?

Desired Behavior

I want the ability to modify offers and get the remaining capacity for an asset (used in sellSideStrategy.go).

Impact

The desired behavior will allow me to easily have the sdex.go implement the exchange.go interface

Feature Suggestion

We can achieve the desired behavior by expanding the exchange.go interface

References

See the calls to sdex. in sellSideStrategy.go

Desired Behavior

I want to be able to cover my positions by offsetting my trades on SDEX against the backing exchange when using the mirror strategy.

Impact

The desired behavior will allow me to run the mirror strategy without worrying about taking on any additional asset inventory risk.

Feature Suggestion

We can achieve the desired behavior by watching for trades against the account and turning around to sell it on the backing exchange.

References

This feature does not exist anywhere else that I'm aware of.

Specification

• fill handler framework
• sleep timer between checking for trades
• fill handlers should run in a separate thread from the watcher thread
• fill handler for mirror strategy

Desired Behavior

I want to be able to run only a single iteration of a given strategy and then have the bot exit

Impact

The desired behavior will allow me to delete offers with a single command rather than having to watch the bot and kill it once it's done.

Feature Suggestion

We can achieve the desired behavior by having a new command line param that takes in the number of iterations we want to run the bot with and have the bot automatically exit once it's hit this number.

Desired Behavior

I want to be able to use historical data (offers, OHLC, etc.) to add some intelligence into my strategies.

Impact

The desired behavior will allow me to build better strategies.

Feature Suggestion

We can achieve the desired behavior by building in the data collection into the framework, allowing the strategies to have access to this data. This will ensure that there is a streamlined process to manage the data.

Desired Behavior

I want to be able to run some basic tests in travis to avoid regressions during code merges

Impact

The desired behavior will allow me to be more confident about code changes and will encourage more testing around the code

Feature Suggestion

We can achieve the desired behavior by adding a line in the travis build that will run tests. Any tests that cannot be run in an automated manner (require API keys or HTTP access) can be set to a mode that will not run in the Travis build.

Additional context

related to #51

Desired Behavior

The bot should have a mode where it tries to only make orders that do not cross any existing orders (maker only orders).

Impact

This can help you avoid unnecessary wash trades if someone else is also market-making but your price feeds or preferences are causing you to place slightly different orders.

Feature Suggestion

Without the DEX protocol itself supporting maker-only orders, anything done in the bot will be a best effort. However, I believe that the basic flow is simply: each time re-evaluating orders to place:

• first load all orders for the asset pair from the DEX
• check if the order you'd be placing will cross with an existing order. If it would, do not place it.

References

Bots aimed at centralized exchanges will generally have this feature because centralized exchanges usually price maker vs. taker orders quite differently.

Example: https://github.com/DeviaVir/zenbot#strategies

Additional context

I believe this should be a flag/parameter for buysell. There's an argument that it could be its own strategy, but I think it's better as a parameter because it might apply to various different strategies (could apply to sell as well, maybe even to balanced).

Specification

(updated by @nikhilsaraf: flag should be optional and in the strategy config file but execution by the framework, augment the strategy interface to support this. This should be handled by either the framework or the exchange implementation but the framework can do the threading of the flag. For the SDEX implementation we can filter out orders by querying the orderbook to find and delete the intersection between existing orderbook and orders we want to create)

Feature Suggestion

We can achieve the desired behavior by creating a layer of filters and each exchange can pass it's own filters which is based on the OrderConstraints

References

This feature does not exist anywhere else that I'm aware of.

Specification

• mirrorStrategy should filter out orders that do not pass the minimum order size requirement
• when offsetting trades it may not be able to offset trades that are below the threshold (even when we filter, partial trades will be less than the threshold)

Desired Behavior

I want to be able to monitor the bot to know when it's running and when it's not

Impact

The desired behavior will allow me to run bots more reliably

Feature Suggestion

We can achieve the desired behavior by starting up a web service whenever the bot starts up that responds with a "success" response to an incoming HTTP message as a starting point. The bot should crash after deleting all the offers on the account if this new service is ever down.

Additional context

Some alternatives that would achieve the same result are:

• use a database that the bot writes to at every time interval

The feature suggested above was the better than these alternatives because a web service is more easily accessible by an external monitoring platform such as Pingdom, or Uptime Robot.

Describe the bug

The bot outputs files for windows without the .exe extension.

Expected behavior

I think the correct behavior in this situation is for the build script to use the .exe extension for the windows platform.

Frequency

The frequency is: Always

Steps To Reproduce

Here are the steps to reproduce the issue (see attachments in section below):

1. Run the build script on a windows platform, or run it with the -d tag on a tagged release
2. see the output for a windows build -- it does not have any extension

Possible Solution

This bug can be fixed by updating scripts/build.sh to add an extension for the windows platform

Your Environment

GOOS: windows

Desired Behavior

I want the logs to be more intuitive and easy to read.

Impact

The desired behavior will allow me to more easily gauge what the bot is doing and will also help with retrospective inspection of specific strategies.

Feature Suggestion

We can achieve the desired behavior by:

1. having structured log lines at the sdex.go level that are well-documented.
2. using a logging framework that supports colors:

• https://github.com/op/go-logging
• https://github.com/dty1er/kubecolor/blob/main/color/color.go

This is a subset of the following issue: #11

This only requests support for a PriceFeed to work with the CCXT library.

See the original issue for more details around this integration

Describe the bug

The bot should follow go convention when naming config variables in .go files.

Expected behavior

Config variables should be changed to use camelCase formatted names; the config files should remain the same in ALL_CAPS format.

Frequency

The frequency is: Always

Steps To Reproduce

N/A

Possible Solution

This bug can be fixed by using double-quoted formatting near config structs.

Your Environment

version: v1.0.0-rc1
git hash: 337f478b0e7a1dc235aef31788754bc1ab11b6a1
build date: 20180813T231816Z
GOOS: linux
GOARCH: amd64

Context

I am unable to look at errors in the code with the noise from these formatting errors because of this bug.

Attachments

No relevant attachments.

Desired Behavior

I want to be able to use the exchange.go interface for trading without running into precision issues. This requires each exchange to be aware of it's precision requirements on a per-trading-pair basis.

Impact

The desired behavior will allow me to trade with kraken, binance, etc. without running into precision issues, enabling usage of the mirror strategy's offset_trades feature.

Feature Suggestion

We can achieve the desired behavior by possibly expanding the exchange interface to return the precision for a given trading pair for price and volume (since they can be different like for Kraken).

References

I think this feature exists in other projects but I have not looked for a reference link.

Additional context

Some alternatives that would achieve the same result are:

• the exchange automatically "fixes up" orders before placing based on the precision requirements, i.e. do not update the exchange.go interface

The feature suggested above was the better than the alternative because by exposing the method in the exchange interface we are letting the framework be aware of precision requirements rather than having this be an implicit operation which can lead to immense confusion.

Desired Behavior

I want to be able to visualize all the Kelp processes that I have running on my system with controls for each instance.

Impact

The desired behavior will allow me to scale up my usage of Kelp and better manage markets.

Feature Suggestion

We can achieve the desired behavior by having a Dashboard UI for Kelp. This can be a web page that interfaces with a server that controls various bot instances running on the machine.

References

This feature exists in quite a few trading bots (both crypto and non-crypto). Please see the specification section below for details.

Specification​

v1.0 (MVP)

1. list of all the bot instances (markets) you are running on your machine
2. create and delete bot instances
3. start and stop bots
4. delete offers for a market

Mockup

Items

• bundle in React.js applications for frontend
• deployment mode embeds JS assets into binary (when building with ./scripts/build.sh -d)
• local mode running from static files in filesystem (when building with ./scripts/build.sh and running with ./bin/kelp server)
• dev mode with hot-reloading of JS code from filesystem (when building with ./scripts/build.sh and running with ./bin/kelp server --dev)
• design mockups
• frontend react components to be used
• list, start, stop/delete bot instances
• edit bot instances
• delete bot instances
• create bot instances (buysell strategy only for now)
• edit form validation
• add trustlines on account when saving
• first pass at polishing

Desired Behavior

The bot should incorporate the liabilities change from Stellar Core (CAP-0003)

This will prevent the bot from placing "phantom" orders on the orderbook, limiting the amounts of the order to the current balance of the asset minus the sum of the amounts in currently open orders.

Impact

The desired behavior will allow me to conform to the new protocol changes and will prevent the bot from being caught off guard when these changes do go into effect in stellar-core.

Feature Suggestion

We can achieve the desired behavior by first summing up the outstanding orders as an initial calculation for liabilities. Once horizon has full support for liabilities we can switchover to using those exposed fields.

References

This feature does not exist anywhere else, please see the specification section below for details.

Additional context

See the Reddit post announcing the liabilities change and the migration schedule to protocol 10, which includes the liabilities change.

Some alternatives that would achieve the same result are:

• Wait for horizon to get full support for liabilities and then update the bot. This would prevent us from having to compute the liabilities on an account directly.

The feature suggested above was better than these alternatives because we are already fetching all the relevant order information from Horizon and summing up our liabilities for XLM, we just need to extend this logic to non-native assets.

Horizon does not fully support liabilities yet, it has a 0 value for all liabilities, so this solution will allow us to be compliant in the mean time.

Specification

CAP-0003: https://github.com/stellar/stellar-protocol/blob/master/core/cap-0003.md

This section of the specification is particularly important:

• account.sellingLiabilities
  • For account A: the amount of native asset offered to be sold, aggregated over all offers owned by A
• account.buyingLiabilities
  • For account A: the amount of native asset offered to be bought, aggregated over all offers owned by A
• trustline.sellingLiabilities
  • For account A and non-native asset X: the amount of X offered to be sold, aggregated over all offers owned by A
• trustline.buyingLiabilities
  • For account A and non-native asset X: the amount of X offered to be bought, aggregated over all offers owned by A

Horizon will only show a valid value for liabilities if the latest ledger (/ledger) is using protocol version 10 (which depends on voting by the network). This should be taken into consideration if using the liabilities values from horizon.

// see sample feature request here: #2

Desired Behavior

// A succinct description of what you are trying to achieve that is not currently supported.
// This can be a request for a new plugin too (strategy, price feed, exchange).
// If your feature request is related to a problem with an existing feature then please submit a bug report instead.
// If this feature does not exist anywhere else then please restrict details to the Specification section below.

i want to spec private keys in environment variables and have script refer to that for specifying. Otherwise this is a disaster to have private keys in github.

Impact

// A succinct description of why you want the desired behavior specified above.

trading and source secret seed should allow me to reference an environment variable name, rather than just a string.

Feature Suggestion

// A succinct description of how you want to achieve the desired behavior.
// If you have looked at the code and know exactly what code changes are needed then please consider submitting a pull request instead.
// If this feature does not exist anywhere else then please restrict any details to the Specification section below.
trading and source secret seed should allow me to reference an environment variable name, rather than just a string

Describe the bug

The bot can exhaust it's XLM balance when running the balanced strategy because the strategy includes the minBalance in it's available capacity to sell the asset when in actuality that balance is not available to sell.

Expected behavior

I think the correct behavior in this situation is for the bot to exclude the minBalance for XLM when computing the balance in the balanced strategy.

Frequency

The frequency is: sometimes (when the XLM balance is close to the minBalance)

Steps To Reproduce

Here are the steps to reproduce the issue (see attachments in section below):

1. Run the balances strategy with a balance close to 23 XLM
2. Using another account, take the offer to sell XLM in exchange for the second asset
3. This puts the balance of the bot below it's minBalance, getting it stuck.

Possible Solution

This bug can be fixed by including min balances in the balanced strategy when computing available balances for pricing and order placing.

Your Environment

version: v1.0.0-rc3
git hash: c8363e7
build date: 20180929T155837Z
GOOS: Linux
GOARCH: arm

References

https://stellar.stackexchange.com/questions/1777/kelp-error-placing-sell-orders/1780

Desired Behavior

The bot would be able to know what initial center price to set by another method than balance calculation.

Impact

This would allow users to set up the balanced strategy without minute fiddling with account balances. It could also help address the concern from #24.

Feature Suggestion

Add the ability for the user to manually set the initial middle price.

Additional context

Another option would be to have the bot read the current SDEX offers and set the center price between the top buy and bottom sell. Doing it that way would only allow the user to consider the extant price fair or use the bot's current balance-based mode; a manual price setting is more flexible.

Specification

1. Add a parameter to the balanced config file where the user would enter their starting middle price
2. Alter the calculation code to allow import of the config price value (for first run of the bot session only)
3. Add a command line flag to activate this behavior

I'll try coding this out but may not be able to figure out the calculation modification without breaking the existing method.