icook / pynode2 Goto Github PK

A rework of Jeff Garzik's PyNode

License: MIT License

Python 100.00%

pynode2's Introduction

PyNode2

Still very green, although contributions are more than welcome

This is a cleanup/rewrite of the pynode written by Jeff Garzik. The goal is to make an easy to work with Bitcoin (or altcoin) node implementation for network analysis in the spirit of Bitnode's getaddr. Working with Python tends to be much quicker/easier than C++ or similar systems languages.

Note that this node implementation isn't designed to be used for real sending/recieving, but more as a malleable tool for education and analysis of the network.

Only Python 2.7 is currently supported due to reliance on Gevent.

Contributing

Contributions are always welcome, my only request at the moment is that you try and code pretty closely to PEP8 standards.

Most tasks on this project will need a pretty good understanding of both Python and Bitcoin internals until there is more documentation.

Running PyNode

For development, setup a virtual enviroment like so:

mkvirtualenv pynode
git clone https://github.com/icook/pynode2
cd pynode2
pip install -r requirements.txt
pip install -e .
pynode example.toml

Currently there are no more automated ways to install. This may change if there's interest.

Current Featureset

Download and synchronize a blockchain (slower than reference wallet)
Connect to peers using DNS seed nodes
Multiple network configurations possible (testnet, litecoin, etc)

pynode2's People

Contributors

Stargazers

Watchers

Forkers

yimai-io herolf

pynode2's Issues

Add examples

Can you please add some python examples that use your library?
It will make it much easy to understand how to use it.

Thanks!
(what I need it to do is download the mempool from a given node)

Abstract storage/retrieval of chaindb

Pulling out all the store/retrieve calls to their own datastore class would be helpful in testing.

Index UTXOs by address that can spend

Main use case is exposing an API for retrieving UTXOs for a specific address. Allows lightweight wallets to operate.

Setup a lightweight mode

Really three different runmodes desired:

Ultralightweight - only accept and store headers, ignore transactions completely. Use case is to connect to a ton of peers and determine the chain that they're on. This allows holistic network status view by seeing all current chains and what % of the network is on each one.
Lightweigt - store headers and UTXOs. All transactions will still need to be downloaded, but disk usage will be greatly minimized.
Standard full node, already implemented.

Fix this

2014-11-10 21:07:08,153 [NodeConn] ERROR: Connection to peer crashed! 162.243.132.6:18333, ver 60002
Traceback (most recent call last):
  File "/home/isaac2/programming/pynode2/pynode/node.py", line 95, in _run
    self.got_data()
  File "/home/isaac2/programming/pynode2/pynode/node.py", line 137, in got_data
    self.got_message(t)
  File "/home/isaac2/programming/pynode2/pynode/node.py", line 265, in got_message
    self.chaindb.putblock(message.block)
  File "/home/isaac2/programming/pynode2/pynode/chain_db.py", line 632, in putblock
    if not self.putoneblock(block):
  File "/home/isaac2/programming/pynode2/pynode/chain_db.py", line 554, in putoneblock
    core.CheckBlock(block)
  File "/home/isaac2/programming/python-bitcoinlib/bitcoin/core/__init__.py", line 734, in CheckBlock
    nSigOps += GetLegacySigOpCount(tx)
  File "/home/isaac2/programming/python-bitcoinlib/bitcoin/core/__init__.py", line 687, in GetLegacySigOpCount
    nSigOps += txout.scriptPubKey.GetSigOpCount(False)
  File "/home/isaac2/programming/python-bitcoinlib/bitcoin/core/script.py", line 879, in GetSigOpCount
    for (opcode, data, sop_idx) in self.raw_iter():
  File "/home/isaac2/programming/python-bitcoinlib/bitcoin/core/script.py", line 724, in raw_iter
    raise CScriptTruncatedPushDataError('%s: truncated data' % pushdata_type, data)
CScriptTruncatedPushDataError: PUSHDATA(49): truncated data

Run a thread that manages node connections

Right now we just connect to the seed nodes and leave it at that. A thread to manage node connections should be added.

Responsibilities might roughly include:

Detecting timeout on current connections (could also be handled in main node code, not sure)
Connecting to other nodes as needed to reach some level

Some possible optimizations that could be done on node connection management:

Try to maintain some small number of rotating connections that we will briefly connect to in order to map the network, find ping times, geolocate, etc. IE, we occasionally disconnect from some in this pool and try finding new connections to replace them, and while connected we don't really run getblocks and mempool.
Select a more diverse set of peers based on geolocation and ping time. This will aid in network propogation.
Actively track blockheight/hash of peers and try to maintain at least X number of peers that are on the same blockheight. In the bad Darkcoin forks there were issues with the clients maxconns getting filled with clients that were forked.