sr-gi / bitcoin_tools Goto Github PK
View Code? Open in Web Editor NEWPython Bitcoin tools
License: BSD 3-Clause "New" or "Revised" License
Python Bitcoin tools
License: BSD 3-Clause "New" or "Revised" License
Implement decompress_pk in bitcoin_tools/core/keys
Hello,
I am trying to use python bitcoin tools 0.2.3 package but it shows errors while installing i got the below error.
But in the requirement package of source contains there packages - ecdsa
python-bitcoinlib
base58
qrcode
Pillow
plyvel
matplotlib
numpy
ujson
ERROR: Failed building wheel for ujson
Failed to build ujson
ERROR: Could not build wheels for ujson which use PEP 517 and cannot be installed directly.
Every package which are mentioned in the requirement is installed properly except ujson and plyvel. Ujson rewuired greater version How to overcome these errors.
Donyou know if it s possible to use the tool to extract addresses from litecoin core?
STATUS was developed having in mind that any kind of computer may be able to run it (even low RAM laptops), since we wanted to allow any user to run analysis over the UTXO set, no matter their hardware capabilities.
However, such design decision makes STATUS work using temporary json
files, which slow down the whole analysis process (due to I/O and having to dump/load from json). Therefore, a server version able to run storing all the intermediate values in memory will be quite useful to speed up analysis process.
I cloned the project, switched to the dev branch and then when I tried to execute: python run_analysis.py I received:
ImportError: No module named bitcoin_tools.analysis.status.data_dump
Probably a noob python issue.
Looks like ujson installation fails when compiled with gcc5+.
Find a solution or an alternative for ujson.
Related to #12 and ultrajson/ultrajson#237
Please help me !
Hi, and thanks for creating the tool.
I am trying to run the example shown on the readme page to parse the UTXO database.
Unfortunately, I was getting empty files as results. I was only able to get something in the file utxos.txt
after changing prefix=b'c'
to prefix=b'C'
in this line in utils.py
After that, the file utxos.txt looks like this:
{"value": "81e14332003bf58224ccd41bd4c94bf0d4a711fc4e1187887c", "key": "4300000136ea56f3b0ee8fbb13059cbd4521ba38074fd9832a86f4335cd7d2267c00"}
{"value": "bf8d427b007ce9617e10685f9aa2f11d19272b0692c425bc83", "key": "43000007608f08c437696f0e12bc50927ac343e48c51f4b9c4a52a484998ad188902"}
{"value": "efd20680e52900df59f7cfb19c60b7e7d4087b15ca5612523d5c53", "key": "43000009c5d6812fe6fd011c921e566bc396d840293efc1c570777d95359fca78400"}
{"value": "d7ae4807016afa6ce4bcea3bfdb9cdcc1818fdf272e695854f", "key": "4300000b99fb3d107e8573bfad3d28b52087e6fb685b156777c73157fa452ba96000"}
{"value": "989160808027000b4d2d0c2252ede24fd76f0d242903d5758697b8", "key": "430000104fbfb34ef6c699cf1ec1f0f8e8da9c6bb71d81ca1c2e449529c86b815c00"}
Does this make sense?
When I later try to run utxo_dump(f_utxos, f_parsed_utxos)
on this file, I get the error:
Traceback (most recent call last):
File "leveldb_dump.py", line 10, in <module>
utxo_dump(f_utxos, f_parsed_utxos)
File "/home/amanusk/Code/bitcoin_tools/bitcoin_tools/analysis/leveldb/data_dump.py", line 48, in utxo_dump
utxo = decode_utxo(data["value"])
File "/home/amanusk/Code/bitcoin_tools/bitcoin_tools/analysis/leveldb/utils.py", line 201, in decode_utxo
data, offset = parse_b128(utxo, offset)
File "/home/amanusk/Code/bitcoin_tools/bitcoin_tools/analysis/leveldb/utils.py", line 112, in parse_b128
more_bytes = int(data, 16) & 0x80 # MSB b128 Varints have set the bit 128 for every byte but the last one,
ValueError: invalid literal for int() with base 16: ''
This happens even if I try to parse a single line of the file
I am running the latest version of bitcoin-core (Perhaps an older version is needed for the tools to work?). Is there anything special that needs to be done when starting bitcoind?
Any help is appreciated.
I've tried installing on a couple of distros (including my usual ubuntu environment and a reasonable-looking centos), but keep running into strange build/architecture issues.
An odd question maybe, but: which linux distribution are you using with this project? I'd like to tinker with the tools a bit, and it seems easier at this point to just use whatever OS you're using than work further on debugging the build/installation problems…
Thanks! ❤️
does this library support segwit transaction? I mean can I create a segwit trx using this library?
Problem is that if you dump utxo set with unspent amounts and sum unspent amounts up it appears that currently it is around 20.600.000 BTC. But according to Glassnode and other blockchain parsing companies it should be around 19.200.000 BTC. I have tried to dump utxo's at different block heights, but bitcoin_tools is constantly giving me higher values.
It would be nice to understand what is cousing this increase in unspent amounts.
EDIT:
I can see that if I dump utxo for 01.03.2009 then majority of utxo have 59 btc balance. But it should be 50, same as block reward.
I tried to parse utxo of litecoin. But apparently the prefix is different, do you happen to know the prefix of litecoin?
Looking at the output of these chainstate tools, I see that the "data" key is the hash160 of the utxo.
I'm curious about the key/tx_id value, e.g. "000000f03ecc02c029184f5dbab0051d1820ecab35b4c8b62a11272bd4dccfea". It's not a transaction id.
Is it just the leveldb key for that utxo?
Is there a way to get the utxo's actual blockchain transaction id (w/o parsing all the blockfiles)?
Hello,
I am installing all the packages using pip command but still i am facing the below error.
Traceback (most recent call last):
File "C:\Users\GSK\AppData\Local\Programs\Python\Python37-32\mybit.py", line 1, in
from bitcoin_tools.core.keys import generate_keys, store_keys
ModuleNotFoundError: No module named 'bitcoin_tools.core'
How can i overcome this error. Please guide me.
Hi,
getting the last block from the chainstate fails for several chainstate heights. One example is height 176760 with the block hash 0000000000000a9569c34dd2f69d39b9763706991418b6ea3836c2b66fb57700. get_chainstate_lastblock calls deobfuscate_value where the check "assert len(value) == len(r)" fails.
I think the if statement above this check is wrong, as an arbitary number of leading zeros could be dropped and not only one.
Steps to replicate this issue:
bitcoin-cli invalidateblock 0000000000000056c50dbea31d9ad6f429aaad61ca313e72e4c00b53fd9fb3fb
(invalidate block 176761)PS: Thanks a lot for the bitcoin tools, they helped me a lot!
When parsing the leveldb both the tx_id
and the the utxo key
are stored in the parsed_utxos.json
file. key
is stored to allow reordering the file when performing the tx_based_analysis
in order to aggregate all the data related to each transaction (since all data can not be loaded in memory due to how much RAM is required). A more optimal way should be found to do so, since the current way has a significant overhead both in terms of data storage and time of execution.
using this repo ?
Thanks
# python run_analysis.py
Traceback (most recent call last):
File "run_analysis.py", line 1, in <module>
from bitcoin_tools.analysis.status.data_dump import transaction_dump, utxo_dump
File "/usr/lib64/python2.7/bitcoin_tools/analysis/status/data_dump.py", line 3, in <module>
from bitcoin_tools.analysis.status.utils import check_multisig, get_min_input_size, roundup_rate, check_multisig_type, \
File "/usr/lib64/python2.7/bitcoin_tools/analysis/status/utils.py", line 3, in <module>
import ujson
ImportError: /root/.local/lib64/python2.7/site-packages/ujson.so: undefined symbol: Buffer_AppendShortHexUnchecked
What am I doing wrong?
Hello there. I am trying to dump the UTXOs using the example code in the README.md:
from bitcoin_tools.analysis.status.data_dump import utxo_dump
from bitcoin_tools.analysis.status.utils import parse_ldb
# Set the version of the Bitcoin Core you are using (which defines the chainstate format)
# and the IO files.
f_utxos = "decoded_utxos.txt"
f_parsed_utxos = "parsed_utxos.txt"
# Parse all the data in the chainstate.
parse_ldb(f_utxos)
# Parses transactions and utxos from the dumped data.
utxo_dump(f_utxos, f_parsed_utxos)
# Data is stored in f_utxos and f_parsed_utxos files respectively
When it gets to the utxo_dump()
function, the script exits with the following error:
Traceback (most recent call last):
File "utxodump.py", line 14, in <module>
utxo_dump(f_utxos, f_parsed_utxos, "bitcoin")
File "/usr/lib/python2.7/bitcoin_tools/analysis/status/data_dump.py", line 101, in utxo_dump
raw_np = out["amount"] / float(min_size)
ZeroDivisionError: float division by zero
I've been able to get around this by hard-coding this in data_dump.py
to try and avoid an error:
# min_size = get_min_input_size(out, utxo["height"], count_p2sh, coin)
min_size = 0.00000001 # my edit
This is just a hack though, and was wondering if there was another reason why I might be getting this error?
This does not work in python 3.6, can it refresh?
ValueError: path 'C:\Users\Administrator/bitcoin_tools/' cannot end with '/'
Failed building wheel for python-bitcoin-tools
Hello i m trying to use it on Windows but it seems donna work. Can you help? Thanx in advance. Regards
I was looking at utxo_dump function. I am trying to understand the utxo database.
Can you specify the 6 standard utxo types ?
# Standard UTXO types std_types = [0, 1, 2, 3, 4, 5]
I thought there are only like 4-5 of them ? P2PKH, P2PK, P2MS, P2SH ?
also, based on the output, is it correct that:
0 --> P2PKH
1 --> P2PK
2 --> P2SH
how about 3,4,5 ?
Thanks
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.