btcsuite / btcsim Goto Github PK

License: ISC License

Go 97.80% Shell 2.20%

btcsim's Introduction

btcsim

btcsim implements a simulation test driver using the simnet network provided by btcd. It launches the required nodes, manages them, runs the simulation and cleans up when done, all by itself, so there's no manual setup/teardown involved. btcsim uses btcrpcclient to interface with the RPC servers.

The input is read from a CSV file with the following headers:

| block height | minimum utxos required | transactions required |

Example:

20000,40000,20000
20001,60000,30000

Based on the input, the minimum required number of utxos are generated by creating transactions using the existing utxos and dividing them until the required number is reached.

The required number of transactions are generated by simulating transactions between multiple actors and using the utxos generated previously.

Mining is controlled using the RPC clients to ensure that the blocks are generated only when required i.e. when the minimum required number of utxos and transactions are created.

Components

Node

This is the first btcd node that is launched. It acts as the server for all Actors and as a peer for the miner.

Actor

An Actor simulates a wallet "Agent" by launching a btcwallet instance which connects to the node server. It is responsible for generating transactions among itself as well as other actors. Multiple actors can be launched simultaneously to simulate a large load due to a heavy multi-user system.

Miner

The Miner launches a btcd instance and simulates a mining node. It is connected to the node server using the addnode RPC call. It is responsible for collecting transactions and mining them when required.

Installation

btcsim depends on btcd and btcwallet, so install those first

$ go get github.com/btcsuite/btcd

$ go get github.com/btcsuite/btcwallet

Now install btcsim

$ go get github.com/btcsuite/btcsim

Usage

Invoking the command btcsim without any flags initializes a single actor simulation, using a default linear curve of 0-10,000 transactions per block:

$ btcsim

For more options, see:

$ btcsim --help

Debugging

To debug btcsim itself, you can turn on profiling using the --profile flag. To debug the processes spawned by btcsim i.e. btcd and btcwallet, you can check the logs written to ~/.btcsim/.

License

Package btcsim is licensed under the copyfree ISC License.

btcsim's People

Contributors

Stargazers

Watchers

btcsim's Issues

miner - the client has been shutdown

Sometimes the miner rpc client has an unexpected shutdown which leads to a panic. Trying to find out the conditions to repro this..

2014/10/13 15:34:40 miner: Cannot stop mining: the client has been shutdown
panic: runtime error: close of closed channel

goroutine 99 [running]:
runtime.panic(0x9018a0, 0xd52335)
        /usr/local/go/src/pkg/runtime/panic.c:279 +0xf5
main.(*Communication).Communicate(0xc20802a620, 0xc2080e3b00, 0xc208120090, 0xc2080e3b30, 0x5, 0x5)
        /home/tuxcanfly/Work/conformal/src/github.com/conformal/btcsim/comm.go:420 +0xed
created by main.(*Communication).Start
        /home/tuxcanfly/Work/conformal/src/github.com/conformal/btcsim/comm.go:141 +0x5e0

Handle corner case where every actor unexpectedly quits

Right now this is not handled at all, eg. in an one-actor simulation if the actor errors at some point, main will block or panic depending on where it is executing at the time the actor encountered the error.

Update to work with new wallet address manager

Need to update btcsim to make it work with the new address manager. Will involve automating the initial wallet setup process.

tx err: invalid amount

Use one exit channel

Right now we are using three different channels for exiting btcsim (stop, fail, interrupt) which is cumbersome especially when using nested selects. Instead, one single channel shall be used which will pass the reason of exiting.

i/o timeouts

In some simulations, especially as the number of actors used is increasing, i/o timeouts may happen to the btcwallet rpc clients.

panic on wallet connection timeout

if maxconnretries is set to lower than default of 30, the connection to wallet times out and causes a panic, need to gracefully handle this.

2015/03/11 23:59:41 actor-18557: Cannot start actor: connection timeout
2015/03/11 23:59:41 node: Shutdown
2015/03/11 23:59:42 miner: Cannot start mining: -32603: No payment addresses specified via --miningaddr
panic: runtime error: close of closed channel

-22: TX rejected: transaction [...] in the pool already spends the same coins

While running simulations, after the first 15k blocks or so, this kind of error starts to happen, even though actors have a large coinbase behind them. This probably is a btcwallet bug but i am opening it here for now.

Update to use 'generate' RPC

Work is currently underway to implement a new 'generate' RPC which can be used to deterministically generate a specified number of blocks (btcsuite/btcd#362).

Once that is done, btcsim should be updated to use it.

Global Address Book

Provide global address book to sim actor.

Block number in csv file

Currently the block number specified in a csv file [first value of the two] plays no part other than being the first value of a record. There are two options: either leave it as is or have it actively participate in a simulation. In the first case it's much simpler and probably better for running btcsim but it currently accepts csv files that we literately don't care about the one of the two fields ie. half the csv file. In the second case, we will have to add some restrictions about compliance of the block numbers in the csv file (comparison with matureBlock, maxBlocks, every block number must be different from the rest, etc.) which won't be a gain for btcsim other than accepting a well-defined csv only.

better log message for no of tx/utxos we're generating

Replaced dots and plus with a simple X/Y progress bar. Also simplified log statements for generating transactions.

don't panic on missing btcwallet, btcd

missing btcd is already being handled, but missing btcwallet panics when the actor is shutting down and cmd.Process is nil

panic: close of closed channel on interrupt

panic: close of closed channel

goroutine 10015 [running]:
main.func·003()
        /home/tuxcanfly/Work/conformal/src/github.com/btcsuite/btcsim/comm.go:587 +0x6a
created by main.(*Communication).Communicate
        /home/tuxcanfly/Work/conformal/src/github.com/btcsuite/btcsim/comm.go:589 +0xcaf

This happens sometimes when a SIGINT is passed after the simulation starts

Transaction curve hang

When using a transaction curve with big rates and a while after the transactions have already started, btcsim hangs. It looks like there is a problem in syncing between the memory pools of the chain server and the miner. More specifically, i have been using this tx curve. The first four blocks succeed in being mined (with the right amount of txs), then in the fifth block, the chain server mempool accepts all 610 txs but in the miner mempool only around 20-30 txs get accepted. With the current design in btcsim it blocks here

Chain server, miner and btcwallet stack dumps respectively:
http://pastebin.com/E8SUERkC
http://pastebin.com/5aHRJnFK
http://pastebin.com/fu1AC8ZX