UPDATE: the functionality supporting negative integers as hardened subkeys is added in PR #98 and removed altogether in PR #100. Those PRs are mutually exclusive and only one should be merged. My personal recommendation is to merge #100 and close (reject) #98 for the reasons outlined in #97 (comment) below.

Unless I've misinterpreted, according to the docs, the following should be equivalent:

i = 23
bip32_node.subkey(i, is_hardened=True)
bip32_node.subkey(-i)

However, see this line (reproduced with some context below). In short, note the raise ValueError(...) statement prevents submitting a negative index, and also prevents execution of the rest of the if statement:

...
    def _subkey(self, i, is_hardened, as_private):
        """Yield a child node for this node.
        i: the index for this node.
        is_hardened: use "hardened key derivation". That is, the public version
        of this node cannot calculate this child.
        as_private: set to True to get a private subkey.
        Note that setting i<0 uses private key derivation, no matter the
        value for is_hardened."""
        ...
        if i < 0:
            raise ValueError("i can't be negative")  # <-- BRAIN DAMAGE(?)
            is_hardened = True
            i = -i
            i |= 0x80000000
        ...
...

I'm trying to write a python code to spend from the generated P2SH address and I've been getting this error: ValueError: p2sh_lookup not set

I then tried running it with the following command (tx script) and got the same error.

tx -a -i 33byJBaS5N45RHFcatTSt9ZjiGb6nK4iV3 -F 0 -f ./wifs.txt -u 1shaYanre36PBhspFL9zG7nt6tfDhxQ4u

signing...
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.7/bin/tx", line 9, in <module>
    load_entry_point('pycoin==0.53', 'console_scripts', 'tx')()
  File "../pycoin-master/pycoin/scripts/tx.py", line 408, in main
    sign_tx(tx, wif_iter(key_iters))
  File "../pycoin-master/pycoin/tx/tx_utils.py", line 170, in sign_tx
    tx.sign(LazySecretExponentDB(wifs, secret_exponent_db))
  File "../pycoin-master/pycoin/tx/Tx.py", line 355, in sign
    self.sign_tx_in(hash160_lookup, idx, self.unspents[idx].script, hash_type=hash_type, **kwargs)
  File "../pycoin-master/pycoin/tx/Tx.py", line 232, in sign_tx_in
    existing_script=self.txs_in[tx_in_idx].script, **kwargs)
  File "../pycoin-master/pycoin/tx/pay_to/ScriptPayToScript.py", line 36, in solve
    raise ValueError("p2sh_lookup not set")
ValueError: p2sh_lookup not set

See here. I'm pretty sure it should be this instead:

def standard_tx_out_script(address, netcodes=DEFAULT_NETCODES):
    script_obj = script_obj_from_address(address, netcodes=netcodes)
    if script_obj is None:
        raise ValueError("can't parse address %s" % address)
    return script_obj.script()

I'm on the road, but I'll submit a patch when I can (unless someone else gets to it first).

I haven't checked, but now that I think about it, I think the code for write_push_data() in tx/script/tools.py will not work well for values that could be encoded into 3 bytes. The script language only supports 1/2/4 byte integers, but the int_to_bytes() function is a little too powerful in that it would encode those in 3 bytes.

the older version I have works fine when I installed the new version the -J errors out

Traceback (most recent call last):
File "/usr/local/bin/genwallet", line 9, in
load_entry_point('pycoin==0.42', 'console_scripts', 'genwallet')()
File "/usr/local/lib/python2.7/dist-packages/pycoin/scripts/genwallet.py", line 85, in main
network="test" if wallet.is_test else "main",
AttributeError: 'Wallet' object has no attribute 'is_test'

I'm not network="test" - FYI

Looks like encoding.py doesn't support testnet wifs and addresses. This is mainly because the allowable_prefixes are only limited to mainnet. Is there any specific reason for this limitation? Or can this limitation be removed to allow testnet?

I don't know if this is me, a bug in pycoin, a bug in bitcoinj, or me:

% JYTHONPATH=.../bitcoinj-0.11-bundled.jar jython
Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06)
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.7.0_51
Type "help", "copyright", "credits" or "license" for more information.
>>> from java.math import BigInteger
>>> from com.google.bitcoin.core import ECKey
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
>>> from com.google.bitcoin.params import MainNetParams, TestNet3Params
>>> priv_key = 16404731722033649474165521611800542240555275746052963990137782680023514762282L
>>> k1 = ECKey(priv_key)
>>> priv_key = BigInteger("16404731722033649474165521611800542240555275746052963990137782680023514762282")
>>> k2 = ECKey(priv_key)
>>> k2.equals(k1)
True
>>> k1.toStringWithPrivate() == k2.toStringWithPrivate()
True
>>> p = MainNetParams.get()
>>> k1.toAddress(p)
1DfSTmkQdmM72hdXtV6fcjnrEeki5dTzv6
>>> k2.toAddress(p)
1DfSTmkQdmM72hdXtV6fcjnrEeki5dTzv6
>>> p = TestNet3Params.get()
>>> k1.toAddress(p)
mtBPkpqPSnnMop79c453Sf1B6eMR1KDMTC
>>> k2.toAddress(p)
mtBPkpqPSnnMop79c453Sf1B6eMR1KDMTC
>>> ^D
% pypy
Python 2.7.6 (394146e9bb673514c61f0150ab2013ccf78e8de7, May 30 2014, 16:16:30)
[PyPy 2.3.0 with GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>> from pycoin.key import Key
>>>> priv_key = 16404731722033649474165521611800542240555275746052963990137782680023514762282L
>>>> k = Key(secret_exponent=priv_key, netcode='BTC')
>>>> k.address()
u'1DkBTgexGMQsXkzfyr3xMdQPiyZpL5KkGV'
>>>> k = Key(secret_exponent=priv_key, netcode='XTN')
>>>> k.address()
u'mtG8kjjw5Nr8JsUHhR2LBYciayAXAEFmDU'
>>>> ^D

But as far as I can tell:
1DfSTmkQdmM72hdXtV6fcjnrEeki5dTzv6
isn't
1DkBTgexGMQsXkzfyr3xMdQPiyZpL5KkGV

Am I missing something?

What's the best way to use pycoin to sign the hexadecimal serialisation of an unsigned transaction, such as may be passed to Bitcoin Core's signrawtransaction API method?

Example input: 01000000014e40a06a3e19165e71351e287346a46f9e768677e64d667bef2857ae175c4b63020000001976a914a0e8d5fe498e6ca2c2000de13fc9cc470bd9b87088acffffffff036c2a0000000000001976a914a0e8d5fe498e6ca2c2000de13fc9cc470bd9b87088ac6c2a00000000000047512103f62c74a1686c41e2639c2c6e9ab336856a405f7aba0b753af6002ce709daf55d211c434e5452505254590000000000000000000000010000000001312d000000000052aededb2b01000000001976a914a0e8d5fe498e6ca2c2000de13fc9cc470bd9b87088ac00000000

Desired output:
01000000014e40a06a3e19165e71351e287346a46f9e768677e64d667bef2857ae175c4b63020000006b48304502210080d73fe780f385c9f1461e62494f4feee39b4c43202a34c4c85471803568489b02206de0ebd1fda2098561e4b64ee8baee1a16a7344e5ab862e332a0c60f8810bd08012103f62c74a1686c41e2639c2c6e9ab336856a405f7aba0b753af6002ce709daf55dffffffff036c2a0000000000001976a914a0e8d5fe498e6ca2c2000de13fc9cc470bd9b87088ac6c2a00000000000047512103f62c74a1686c41e2639c2c6e9ab336856a405f7aba0b753af6002ce709daf55d211c434e5452505254590000000000000000000000010000000001312d000000000052aededb2b01000000001976a914a0e8d5fe498e6ca2c2000de13fc9cc470bd9b87088ac00000000

Thanks for your help.

I was going through the steps of http://blog.richardkiss.com/?p=313, using pycoin as of commit b961cdc under Python 2.7.6 on Debian Linux.

Ran into this error:

$ head -c200 /dev/random | python2.7 genwallet.py - 
Traceback (most recent call last):
  File "genwallet.py", line 129, in 
    main()
  File "genwallet.py", line 41, in main
    args.inputfile = sys.stdin.buffer
AttributeError: 'file' object has no attribute 'buffer'

Debian version details:

$ uname -a
Linux test 3.13-1-amd64 #1 SMP Debian 3.13.10-1 (2014-04-15) x86_64 GNU/Linux

Python version details:

$ python2.7 --version
Python 2.7.6

Given knowledge of an extended public key and one of its publicly-derived private children, an attacker may recover the extended private key. Please add a notice to the README that all extended public keys that are uploaded to insecure servers must be prime children of the root node, and that private children of that pubkey must never be revealed.

See https://github.com/sbuss/bitmerchant#security-warning and http://bitcoinmagazine.com/8396/deterministic-wallets-advantages-flaw/

I was going through the steps of http://blog.richardkiss.com/?p=313, using pycoin as of commit b961cdc under Python 3.4.0 on Debian Linux.

Ran into the error below while trying out the "-i" command line flag:

$ python3.4 ./genwallet.py -f master-private-key -s 0p.pub -i
Traceback (most recent call last):
  File "./genwallet.py", line 129, in 
    main()
  File "./genwallet.py", line 98, in main
    if wallet.is_test:
AttributeError: 'Wallet' object has no attribute 'is_test'

Debian version details:

$ uname -a
Linux test 3.13-1-amd64 #1 SMP Debian 3.13.10-1 (2014-04-15) x86_64 GNU/Linux

Python version details:

$ python3.4 --version
Python 3.4.0+

While testing p2sh multisig I tried instrumenting this function. I noticed the 'is_p2h' clause was failing because it was comparing a char (e.g. script_public_key[0]) against an int (opcodes.OP_HASH160).

repro: "tx -a 485716e53b422aca0fe5b1ded21360695ce5f49255d80e10db56458ed6962ff3"

Reading through the BIP0032 examples, I wanted to try out the Unsecure money receiver: M/i'/0 example.

As much as I understand, M/i'/0 should be the following: use the public key of the master node, do a private-key-derivation wallet (hence the prime), and use the external chain (hence the 0).

If I try to do even the first step towards there, by generating M/0', I think it should be like this (with the test vector 1):

from pycoin import wallet
wM = wallet.Wallet.from_wallet_key('xpub661MyMwAqRbcFtXgS5sYJABqqG9YLmC4Q1Rdap9gSE8NqtwybGhePY2gZ29ESFjqJoCu1Rupje8YtGqsefD265TMg7usUDFdp6W1EGMcet8')
print( wM.subkey(i=0, is_prime=True).wallet_key() )

This breaks down with pycoin.wallet.PublicPrivateMismatchError: can't derive a private key from a public key.

Is this the correct behaviour? Shouldn't M be able to retrieve all the public keys (since in the examples M is an auditor)?

tx 8d852ef1b2d290b515763f604f753e84352059cf31766845eadcb4ac8ab4edbb/0/a9146de2a879701db827dafec3c1f895b35093205d4587/30000 3AAhGP5qDfo3DvAEZ1WeQgp71y4c112NBr -o tx.bin

doesn't work.

Running sudo setup.py install fails with unkonwn distribution option

Python 2.7 —

vagrant@millstone:~/github/pycoin$ python --version
Python 2.7.3
vagrant@millstone:~/github/pycoin$ sudo ./setup.py install
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'entry_points'
  warnings.warn(msg)
running install
running build
running build_py
running install_lib
running install_egg_info
Removing /usr/local/lib/python2.7/dist-packages/pycoin-0.23.egg-info
Writing /usr/local/lib/python2.7/dist-packages/pycoin-0.23.egg-info
vagrant@millstone:~/github/pycoin$ genwallet
genwallet: command not found

Python 3 —

~/github/others/pycoin ∀ python --version
Python 3.3.2
~/github/others/pycoin ∀ sudo ./setup.py install
/usr/lib/python3.3/distutils/dist.py:257: UserWarning: Unknown distribution option: 'entry_points'
  warnings.warn(msg)
running install
running build
running build_py
running install_lib
running install_egg_info
Removing /usr/lib/python3.3/site-packages/pycoin-0.23-py3.3.egg-info
Writing /usr/lib/python3.3/site-packages/pycoin-0.23-py3.3.egg-info
~/github/others/pycoin ∀ genwallet
-bash: genwallet: command not found

Hi there Richard, your pycoin library helped me understand BIP32 to a far extent. Thank you for that.

I'm trying to use BIP32 components between wallets. Specifically pycoin Wallet, Armory and Electrum. Since we're talking about BIP32, I expected this to be possible. But find it hard to get it working.

I tried the following two use cases:

1. use Armory Watch-only to monitor my deterministic wallet made using pycoin
2. generate a Wallet in pycoin from the Electrum seed that I saved on first install

Armory watch-only

Armory offers the import of a paper wallet (HD), which has a format something like this 'abcd efgh tral alal' for both the chain code as the secret exponent.
I did not succeed to convert the pycoin Wallet secret and chain code to the format expected by Armory. The pycoin chain code is a binary blip.

The import of a digital wallet seems to be an Armory specific format.

Is it possible with pycoin to create something that Armory will accept?
Or should we suggest the Armory team to offer imports of values closer to BIP32?

I would be satisfied if it were possible the other way around: to create the deterministic wallet in Armory and then use the secret and chain part in pycoin to generate individual addresses and private keys.

But there I also failed.

Electrum seed

This is analogous to the previous example. How to convert the Electum seed to be used in pycoin to generate addresses and private keys?

I would appreciate it if you could give me a few directions to get this working. Thank you!

Regarding this section of BIP32:
https://github.com/bitcoin/bips/blob/2ea19daaa0380fed7a2b053fd1f488fadba28bda/bip-0032.mediawiki#private-parent-key--private-child-key

In case parse256(IL) ≥ n or ki = 0, the resulting key is invalid, and one should proceed with the next value for i. (Note: this has probability lower than 1 in 2^(127).)

In particular I suspect this should be implemented near:

Alternatively, what about raising an error instead? I know this is not standards-compliant, but if someone really needs that functionality they can just increment the value themselves and try again, I believe.

The ECDSA implementation seems to wrap around 0/generator_secp256k1.order():

>>> from pycoin.ecdsa.secp256k1 import generator_secp256k1
>>> from pycoin.ecdsa import public_pair_for_secret_exponent
>>> pp1 = public_pair_for_secret_exponent(generator_secp256k1, -1)
>>> pp2 = public_pair_for_secret_exponent(generator_secp256k1, generator_secp256k1.order() - 1)
>>> pp1 == pp2
True
>>> pp1 = public_pair_for_secret_exponent(generator_secp256k1, 0)
>>> pp2 = public_pair_for_secret_exponent(generator_secp256k1, generator_secp256k1.order() + 0)
>>> pp1 == pp2
True
>>> pp1 == (None, None)
True
>>> pp1 = public_pair_for_secret_exponent(generator_secp256k1, 1)
>>> pp2 = public_pair_for_secret_exponent(generator_secp256k1, generator_secp256k1.order() + 1)
>>> pp1 == pp2
True

In fact:

>>> for i in range(256):
...   pp1 = public_pair_for_secret_exponent(generator_secp256k1, i)
...   pp2 = public_pair_for_secret_exponent(generator_secp256k1, generator_secp256k1.order() + i)
...   assert pp1 == pp2
...   pp1 = public_pair_for_secret_exponent(generator_secp256k1, -i)
...   pp2 = public_pair_for_secret_exponent(generator_secp256k1, generator_secp256k1.order() - i)
...   assert pp1 == pp2
...
>>>

This is a little misleading, but something we can probably live with. EDIT: For clarification, I mean that given a raw random distribution of potential secret exponents [0..2²⁵⁶), wrapping around at the order point favors the first 0x14551231950b75fc4402da1732fc9bebf keys by 2:1.^[1]

However, there also seems to be a behavioral asymmetry in pycoin.key.Key (which is at the core of this issue):

>>> from pycoin.key import Key
>>> k = Key(secret_exponent=0)
>>> k
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pycoin/key/Key.py", line 222, in __repr__
    r = self.public_copy().as_text()
  File "pycoin/key/Key.py", line 199, in public_copy
    is_compressed=(self._hash160_compressed is not None), netcode=self._netcode)
  File "pycoin/key/Key.py", line 43, in __init__
    raise ValueError("exactly one of secret_exponent, public_pair, hash160 must be passed.")
ValueError: exactly one of secret_exponent, public_pair, hash160 must be passed.
>>> k.public_pair()
>>> k = Key(secret_exponent=generator_secp256k1.order())
>>> k
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pycoin/key/Key.py", line 222, in __repr__
    r = self.public_copy().as_text()
  File "pycoin/key/Key.py", line 198, in public_copy
    return Key(public_pair=self.public_pair(), prefer_uncompressed=self._prefer_uncompressed,
  File "pycoin/key/Key.py", line 123, in public_pair
    if not ecdsa.is_public_pair_valid(ecdsa.generator_secp256k1, public_pair):
  File "pycoin/ecdsa/ecdsa.py", line 125, in is_public_pair_valid
    return generator.curve().contains_point(public_pair[0], public_pair[1])
  File "pycoin/ecdsa/ellipticcurve.py", line 56, in contains_point
    return ( y * y - ( x * x * x + self.__a * x + self.__b ) ) % self.__p == 0
TypeError: unsupported operand type(s) for *: 'NoneType' and 'NoneType'
>>> k.public_pair()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pycoin/key/Key.py", line 123, in public_pair
    if not ecdsa.is_public_pair_valid(ecdsa.generator_secp256k1, public_pair):
  File "pycoin/ecdsa/ecdsa.py", line 125, in is_public_pair_valid
    return generator.curve().contains_point(public_pair[0], public_pair[1])
  File "pycoin/ecdsa/ellipticcurve.py", line 56, in contains_point
    return ( y * y - ( x * x * x + self.__a * x + self.__b ) ) % self.__p == 0
TypeError: unsupported operand type(s) for *: 'NoneType' and 'NoneType'

First, Key.__init__ allows silent creation of a corrupt Key object when passed a bad value for secret_exponent. Second, if secret_exponent is 0, one can extract (without triggering an exception) a None public key from the public_pair() method. However, if the secret_exponent is generator_secp256k1.order(), then the public_pair() method raises an exception.

I'm pretty sure the correct behavior is that Key.__init__ should raise an exception in the constructor when secret_exponent is bad.

Tagging as possibly related to #103.

^[1] Derviation:

>>> distribution_bits = 2 ** 256 - 1
>>> order_bits = 0xfffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141 - 1 # sort of; 0 is still rejected
>>> favored_bits = distribution_bits ^ order_bits
>>> '0x{:x}'.format(favored_bits)
u'0x14551231950b75fc4402da1732fc9bebf'

This is not an issue where secret exponents not in [1, order - 1] are rejected outright. One problem, is where the secret exponent is derived from a seed (i.e., hidden from the caller, e.g., BIP32Node.from_master_secret(...)). This can be checked after-the-fact, however:

>>> k1 = Key(secret_exponent=1)
>>> k2 = Key(secret_exponent=generator_secp256k1.order() + 1)
>>> k1.public_pair() == k2.public_pair()
True
>>> k1.secret_exponent() == k2.secret_exponent()
False
>>> k1 == k2
False
>>> k1.secret_exponent() >= 1 and k1.secret_exponent() < generator_secp256k1.order()
True
>>> k2.secret_exponent() >= 1 and k2.secret_exponent() < generator_secp256k1.order()
False

To put this in perspective, consider:

>>> from decimal import Decimal
>>> Decimal(favored_bits + 1) / Decimal(distribution_bits + 1)
Decimal('3.734455345040134152759582548E-39')

This is related to the error exposed in the commit: ec69967

I tried to check the signature of Gavin Andresen's test multisig transaction:
837dea37ddc8b1e3ce646f1a656e79bbd8cc7f558ac56a169626d649ebe2a3ba
described in his gist: https://gist.github.com/gavinandresen/3966071
but the function is_signature_ok() returns False.

Then I tried to sign a raw transaction Gavin created myself, and the result
does have a valid (according to pycoin) signature, however a different one.
You can check this running the following code:
https://gist.github.com/gitmarek/3593fbd9eecbbbebcdc0

Two questions:

1. How could I sign the raw transaction one private key at the time, as in the Gavin's gist?
   When I try to specify only one key in hash160_lookup dict, I got an error.
2. Is the whole issue a bug in pycoin, or am I doing something wrong?

This may be related to #105 and #106. See this line in vm.py for details, reproduced in pertinent part below:

def eval_script(script, signature_for_hash_type_f, expected_hash_type=None, stack=[],
        disallow_long_scripts=True):
    ...
    try:
        while pc < len(script):
            ...
    except Exception:
        logger.exception("script failed") # [oops, we don't do anything meaningful with the exception!]

    return len(stack) != 0 # [so as long as this is true, the script is valid]

This means that if an exception is raised during the script, a message may be logged, but as long as the item on the top of the stack is nonzero, the script will still be considered valid. 😮

I tried changing the except block to the following:

...
from sys import exc_info
...
def eval_script(script, signature_for_hash_type_f, expected_hash_type=None, stack=[],
        disallow_long_scripts=True):
    ...
    except Exception:
        logging.exception("script failed", exc_info=exc_info()) # log the exception (use the generic logger for now, just to get some output)
        return False # actually return the fact that the script failed

    return len(stack) != 0

But this causes the following test failure:

=================================== FAILURES ===================================
_____________________________ TestTx.test_is_valid _____________________________

self = <tests.bc_transaction_test.TestTx testMethod=test_is_valid>

    def test_is_valid(self):
        for (prevouts, tx_hex, flags) in txs_from_json(TX_VALID_JSON):
            try:
                tx = Tx.tx_from_hex(tx_hex)
            except:
                self.fail("Cannot parse tx_hex: " + tx_hex)

            if not check_transaction(tx):
                self.fail("check_transaction(tx) = False for valid tx: " +
                          tx_hex)

            unspents = [Spendable(coin_value=1000000,
                                  script=compile_script(prevout[2]),
                                  tx_hash=prevout[0], tx_out_index=prevout[1])
                        for prevout in prevouts]
            tx.set_unspents(unspents)

            bs = tx.bad_signature_count()
            if bs > 0:
                msg = str(tx_hex) + " bad_signature_count() = " + str(bs)
>               self.fail(msg)
E               AssertionError: 01000000020002000000000000000000000000000000000000000000000000000000000000000000000151ffffffff0001000000000000000000000000000000000000000000000000000000000000000000006b483045022100c9cdd08798a28af9d1baf44a6c77bcc7e279f47dc487c8c899911bc48feaffcc0220503c5c50ae3998a733263c5c0f7061b483e2b56c4c41b456e7d2f5a78a74c077032102d5c25adb51b61339d2b05315791e21bbe80ea470a49db0135720983c905aace0ffffffff010000000000000000015100000000 bad_signature_count() = 1

tests/bc_transaction_test.py:235: AssertionError
===================== 1 failed, 83 passed in 47.90 seconds =====================
ERROR: InvocationError: '.../pycoin/.tox/pypy/bin/py.test tests'

The failing transaction is from this test case. The de-serialzed transaction is: Tx [b56471690c3ff4f7946174e51df68b47455a0d29344c351377d712e6d00eabe5] (v:1) [TxIn<0000000000000000000000000000000000000000000000000000000000000200[0] "OP_1">, TxIn<0000000000000000000000000000000000000000000000000000000000000100[0] "3045022100c9cdd08798a28af9d1baf44a6c77bcc7e279f47dc487c8c899911bc48feaffcc0220503c5c50ae3998a733263c5c0f7061b483e2b56c4c41b456e7d2f5a78a74c07703 02d5c25adb51b61339d2b05315791e21bbe80ea470a49db0135720983c905aace0">] [TxOut<0 mbtc "OP_1">]. The unspents are: [Spendable<1E+1 mbtc "0000000000000000000000000000000000000000000000000000000000000100:0" 0/0/0>, Spendable<1E+1 mbtc "0000000000000000000000000000000000000000000000000000000000000200:0" 0/0/0>]. The unspent scripts are: [u'OP_DUP OP_HASH160 e52b482f2faa8ecbf0db344f93c84ac908557f33 OP_EQUALVERIFY OP_CHECKSIG', 'OP_1']. The error logged is:

.ERROR:root:script failed
Traceback (most recent call last):
  File ".../pycoin/pycoin/tx/script/vm.py", line 98, in eval_script
    raise ScriptError("VERIFY failed at %d" % (pc-1))
ScriptError: VERIFY failed at 23

Note that line 98 actually refers to line 97 in the original file, since I added a line to import exc_info.

Basically the issue is:

bip32_node.subkey_for_path("1/2/1/2") != bip32_node.subkey_for_path("1/2").subkey_for_path("1/2")

btcutil works the way BIP32 child deriviation should work : http://bitcoin.stackexchange.com/questions/36247/bip32-child-derivation-from-another-child

That class was removed. Great lib btw :)

This might be by design, but I'm pretty sure with the verification (and small change) merged with #40, these methods/functions can be safely expanded to accommodate SIGHASH_SINGLE, etc.

I haven't yet investigated this very deeply. I'm assuming _is_compressed went away at some point?

Traceback (most recent call last):
  ...
  File ".../site-packages/pycoin/key/Key.py", line 198, in public_copy
    is_compressed=self._is_compressed, netcode=self._netcode)
AttributeError: 'Key' object has no attribute '_is_compressed'

I'll see if I can propose something when I have time (but I probably won't be able to get to it for a little while). 😫

With this piece of code, I am still able to get the secret_component:

my_prv = BIP32Node.from_master_secret(SECRET)
uag = my_prv.subkey(i=0, is_hardened=True, as_private=True)
priv = uag.subkey(i=0, is_hardened=False, as_private=False).secret_exponent()
print str(priv)

... This is a bug, am I right?

Here's the list (originally from this #112 (comment)):

• "The following tests for the presence of a bug in the handling of SIGHASH_SINGLE"
• "Empty pubkey"
• "Empty signature" (index 1)
• "Empty signature" (index 2)

These tests are failing, but their failures are being hidden by #112.

@mbelshe discovered a bug in my project, bitmerchant sbuss/bitmerchant#29, that is also present in pycoin.

The issue is that children with numbers greater than 0x80000000 are always treated as non-prime in the subkey_for_path derivation. I think that if the child number is greater than 0x80000000 pycoin should properly treat it as prime, or you can throw an exception if you think that's better.

In [1]: child_number = 0x80000000 + 1

In [2]: from bitmerchant import wallet

In [3]: w = wallet.Wallet.from_master_secret('correct horse battery staple')

In [4]: w.get_child(child_number).to_address()
Out[4]: u'19Z7LffWuJ3mfAJyeyz4HmmyGnyathrMHd'

In [5]: w.get_child_for_path('m/%s' % child_number).to_address()
Out[5]: u'1Na7JXH1Nx71xKY5EHapTo3NKPAwNfoAai'

In [6]: # Oops! That's wrong!

In [7]: from pycoin.wallet import Wallet

In [8]: w = Wallet.from_master_secret('correct horse battery staple')

In [9]: w.subkey(child_number)
Out[9]: private_for <1Na7JXH1Nx71xKY5EHapTo3NKPAwNfoAai>

In [10]: # pycoin doesn't set the prime flag correctly, either

In [11]: w.subkey_for_path('%s' % child_number)
Out[11]: private_for <1Na7JXH1Nx71xKY5EHapTo3NKPAwNfoAai>

In [12]: # yep, same bug :(

With over 290 altcoins in the wild, a single-letter "netcode" is technically insufficient. It's also hard to remember in my opinion. I suggest moving to short multi-letter codes for each network, in the style of ISO currency codes.

BTC - Bitcoin
LTC - Litecoin
XTN - Bitcoin Testnet3

... and so on. Each of the altcoins has a established code which is used on the exchanges. Most are 3 letters, but there are 2 and 4 letter ones as well.

I can submit a pull for this change, if you wish, but I would be tempted to rework the internals of networks.py at the same time.

I havn't looked deeply into the differences but it seems that both Key and Wallet do largely the same thing, ie, allow to instantiate with a public or private key and traverse. Which are we supposed to use?

While I'm at it:

• It'd be nice if Key.subkey accepted integers
• I think there's a mistake in the README; shouldn't "where P is a subkey of P" be "where P is a subkey of K" ?
• Another: "pycoin.bip32.Wallet" should be "pycoin.bip32.key.Wallet"

Cheers

I saw a new version at 0.51 but the repo hasnt been tagged from the commit hash that is 0.51

This worked as of bcdff1f, but fails as of c441ea7 (and still fails as of 412efbc).

2015-02-03T21:36:31.77704   File ".../site-packages/pycoin/tx/Tx.py", line 241, in verify_tx_in
2015-02-03T21:36:31.77705     b2h_rev(tx_in.previous_hash), tx_in_idx))
2015-02-03T21:36:31.77705 pycoin.tx.Tx.ValidationFailureError: just signed script Tx ... TxIn index
 0 did not verify
2015-02-03T21:36:31.77706
2015-02-03T21:36:31.82969    ERROR (15598): script failed
2015-02-03T21:36:31.82972 Traceback (most recent call last):
2015-02-03T21:36:31.82973   File ".../site-packages/pycoin/tx/script/vm.py", line 154, in eval_script
2015-02-03T21:36:31.82973     signature_hash = signature_for_hash_type_f(signature_type, script=script)
2015-02-03T21:36:31.82974 TypeError: <lambda>() got an unexpected keyword argument 'script'

c441ea7 adds a keyword parameter to the call for signature_for_hash_type_f.

Not all things pointed to by that symbol accept a script argument (note the two lambda expressions in Tx.py):

% git co c441ea75885167a809ae402d03e1066ed89d3804
...
% find . -name \*.py | xargs egrep -n 'signature_for_hash_type_f\s*[(=]'
./pycoin/tx/Tx.py:224:        signature_for_hash_type_f = lambda hash_type: self.signature_hash(tx_out_script, tx_in_idx, hash_type)
./pycoin/tx/Tx.py:237:        signature_for_hash_type_f = lambda hash_type: self.signature_hash(tx_out_script, tx_in_idx, hash_type)
./pycoin/tx/Tx.py:339:        def signature_for_hash_type_f(hash_type, script=None):
./pycoin/tx/script/vm.py:121:                signature_hash = signature_for_hash_type_f(signature_type)
./pycoin/tx/script/vm.py:154:                    signature_hash = signature_for_hash_type_f(signature_type, script=script)

Two offenders:

• Tx.py at line 224
• Tx.py at line 237

Only one of them appears to have been addressed in 9333471:

% git co 93334719719a6f11144425030594250432b33840
...
% find . -name \*.py | xargs egrep -n 'signature_for_hash_type_f\s*[(=]'
./pycoin/tx/script/vm.py:121:                signature_hash = signature_for_hash_type_f(signature_type, script=script)
./pycoin/tx/script/vm.py:154:                    signature_hash = signature_for_hash_type_f(signature_type, script=script)
./pycoin/tx/Tx.py:240:        signature_for_hash_type_f = lambda hash_type, script: self.signature_hash(script, tx_in_idx, hash_type)
./pycoin/tx/Tx.py:253:        signature_for_hash_type_f = lambda hash_type: self.signature_hash(tx_out_script, tx_in_idx, hash_type)
./pycoin/tx/Tx.py:354:        signature_for_hash_type_f = lambda hash_type, script: self.signature_hash(script, tx_in_idx, hash_type)

Remaining offender after 9333471:

• Tx.py at line 253

I'm not sure how tests are still passing with this? 😕

Hi,

Could you have a look at your MultiSig implementation?

https://github.com/richardkiss/pycoin/blob/master/pycoin/tx/pay_to/ScriptMultisig.py

I am trying to figure out how to transfer BTC to a multisig address and spending it again to another address, however the ScriptMultisig .address() method does not work. Could you fix it or put me in the right direction?

Thanks in advance,

Machiel

Traceback (most recent call last):
  File "multisig.py", line 31, in <module>
    txs_out=TxOut(2, tx.address()),  # Transfer 2 satoshi to multisig addr
  File "...../pycoin/tx/pay_to/ScriptMultisig.py", line 55, in address
    if self._address is None:
AttributeError: 'ScriptMultisig' object has no attribute '_address'

Please add some usage examples to your readme.

I've been working to make pycoin work with Bitcoin Testnet Multisig, I added Blockr.io testnet API to get the spendables, but the chain of code getting there has not yet been fixed.

This is the code I used to get the testnet spendables:

def spendables_for_addr(address, netcode='BTC'):
    """
    Return a list of Spendable objects for the
    given bitcoin address.
    """
    from pycoin.convention import btc_to_satoshi
    from pycoin.tx import Tx, Spendable
    from urllib2 import urlopen

    #Support for bitcoin testnet api in Blockr.io
    if netcode == "XTN":
        URL = "http://tbtc.blockr.io/api/v1/address/unspent/%s" % address
    if netcode == "BTC":
        URL = "http://btc.blockr.io/api/v1/address/unspent/%s" % address
    r = json.loads(urlopen(URL).read().decode("utf8"))
    spendables = []
    for u in r.get("data", {}).get("unspent", []):
        coin_value = btc_to_satoshi(u.get("amount"))
        script = h2b(u.get("script"))
        previous_hash = h2b_rev(u.get("tx"))
        previous_index = u.get("n")
        spendables.append(Spendable(coin_value, script, previous_hash, previous_index))
    return spendables

Now the problem is with the WIF encoding, I guess the solution would be to pass netcode all along the functions to reach https://github.com/richardkiss/pycoin/blob/master/pycoin/encoding.py#L187 or probably to LazySecretExponentDB(...)

Here is the encoding error:

    tx_final =  tx_utils.create_signed_tx([spendables_for_addr(address, netcode=temp_netcode)[0]], [testnet_address], wifs=([key.wif() for key in keys[:M]]), fee=1000, netcode=temp_netcode, p2sh_lookup=p2sh_lookup)
  File "../pycoin/pycoin/tx/tx_utils.py", line 202, in create_signed_tx
    sign_tx(tx, wifs=wifs, secret_exponent_db=secret_exponent_db, **kwargs)
  File "../pycoin/pycoin/tx/tx_utils.py", line 170, in sign_tx
    tx.sign(LazySecretExponentDB(wifs, secret_exponent_db), **kwargs)
  File "../pycoin/pycoin/tx/Tx.py", line 374, in sign
    hash160_lookup, idx, self.unspents[idx].script, hash_type=hash_type, **kwargs)
  File "../pycoin/pycoin/tx/Tx.py", line 249, in sign_tx_in
    existing_script=self.txs_in[tx_in_idx].script, **kwargs)
  File "../pycoin/pycoin/tx/pay_to/ScriptPayToScript.py", line 41, in solve
    underlying_solution = script_obj.solve(**kwargs)
  File "../pycoin/pycoin/tx/pay_to/ScriptMultisig.py", line 118, in solve
    result = db.get(hash160)
  File "../pycoin/pycoin/tx/tx_utils.py", line 29, in get
    secret_exponent = wif_to_secret_exponent(wif)
  File "../pycoin/pycoin/encoding.py", line 203, in wif_to_secret_exponent
    return wif_to_tuple_of_secret_exponent_compressed(wif, allowable_wif_prefixes=allowable_wif_prefixes, netcode = netcode)[0]
  File "../pycoin/pycoin/encoding.py", line 197, in wif_to_tuple_of_secret_exponent_compressed
    raise EncodingError("unexpected first byte of WIF %s" % wif)
pycoin.encoding.EncodingError: unexpected first byte of WIF cVVNL17sdMoGQasdSdj4jGJHTaz6JohgmS2AahhhatQwRFn

The workaround for now is to comment these two lines https://github.com/richardkiss/pycoin/blob/master/pycoin/encoding.py#L195-L196:

    if actual_prefix not in allowable_wif_prefixes:
        raise EncodingError("unexpected first byte of WIF %s" % wif)

or add [b'\xef'] to allowable_wif_prefixes.

I believe many multisig wallets (CoinVaut, BitGo) use the convention that OP_0 is used as a placeholder for missing signatures. I am not sure if it is a standard, but it does appear to be used in many places. Can ScriptMultisig._dummy_signature please be changed to return "\x00" instead? Thanks.

Periodically when partially signing transactions (e.g., in a multisig situation), one gets these messages:

ERROR:root:script failed
Traceback (most recent call last):
  File ".../.virtualenvs/ve-server-pypy/site-packages/pycoin/tx/script/vm.py", line 157, in eval_script
    ecdsa.generator_secp256k1, signature_hash, sig_pair)
  File ".../.virtualenvs/ve-server-pypy/site-packages/pycoin/ecdsa/ecdsa.py", line 168, in possible_public_pairs_for_signature
    R = ellipticcurve.Point(curve, x, y, order)
  File ".../.virtualenvs/ve-server-pypy/site-packages/pycoin/ecdsa/ellipticcurve.py", line 70, in __init__
    if self.__curve: assert self.__curve.contains_point( x, y )
AssertionError

These are basically harmless under a multisig context, but one cannot suppress them without suppressing other messages logged to the root logger.

See, e.g.:

I want to expand support in blocks.py for non-bitcoin blocks and transactions, specifically:

• Litecoin (et al) blocks have a hash calculated with scrypt not SHA256
• Blackcoin blocks (and likely many other proof-of-stake altcoins) have a few extra fields in the block header, and transaction serializations.

To do this, I need to add a dependency: ltc_scrypt. See github here and pypi

It's a C module, but it's small, focused and doesn't pull in lots of dependences (ie. it doesn't need OpenSSL).

I could simply add it to pip-req.txt or I could leave it out, and print an error on ImportError when it's needed. Not clear to me which approach is better in this case.

sorry, my mistake

The four functions to_bytes and int_to_bytes are similar, as are from_bytes and bytes_to_int. They should be merged.

https://en.bitcoin.it/wiki/List_of_address_prefixes says that “c” MUST be the leading symbol for a testnet WIF in base58.

239     9   Testnet Private key (for uncompressed pubkey)   91eWjgRmucdtYHpMdsHbn9h8UU8hdoMNSKj8p3QAj6VTLyBnjj6
239     c   Testnet Private key (for compressed pubkey)     cNJFgo1driFnPcBdBX8BrJrpxchBWXwXCvNH5SoSkdcF6JXXwHMm

while we can observe that pycoin.wallet fails to recognize testnet while generating WIF representation for the private key —

vagrant@millstone:~/bip0032$ genwallet -tu > mk
vagrant@millstone:~/bip0032$ genwallet -wf mk | head -c 2
L2

From https://bitcoin.org/en/developer-reference#term-op-checkmultisig:

Because public keys are not checked again if they fail any signature comparison, signatures must be placed in the signature script using the same order as their corresponding public keys were placed in the pubkey script or redeem script. See the OP_CHECKMULTISIG warning below for more details.

Right now, pycoin behaves like this:

# start with tx1 (not shown) which has one TxIn that consumes
# a 2-of-2 multisig with keys k1 and k2 (in that order)

# make a copy of tx1 for comparison
tx2.tx_from_hex(tx1.as_hex())
tx2.set_unspents(tx1.unspents)

# sign tx1 in one order
tx1.sign(k1)
tx1.sign(k2)

# sign tx2 in the reverse order
tx2.sign(k2)
tx2.sign(k1)

# check some stuff
assert tx1.is_signature_ok(0) # works
assert tx2.is_signature_ok(0) # works (should fail if the next assertion fails)
assert tx1.txs_in[0].script == tx2.txs_in[0].script # fails (should work if the previous assertion works)

# an inspection of tx2.txs_in[0].script will reveal that the
# order of the signatures is inverted from that of
# tx1.txs_in[0].script

Either the last two assertions should both fail, or both work, but only the first works right now (which is the bug).

if a transaction is created with tx_util.create_tx() and saved with tx.as_hex(include_unspents=True) it would include the spendables, however to read it again with tx.tx_from_hex(tx_hex) it fails to read the spendables

#pycoin/tx/Tx.py
    def parse(class_, f):
        """Parse a Bitcoin transaction Tx from the file-like object f."""
        version, count = parse_struct("LI", f)
        txs_in = []
        for i in range(count):
            txs_in.append(TxIn.parse(f))
        count, = parse_struct("I", f)
        txs_out = []
        for i in range(count):
            txs_out.append(TxOut.parse(f))
        lock_time, = parse_struct("L", f)
        return class_(version, txs_in, txs_out, lock_time)

    @classmethod
    def tx_from_hex(class_, hex_string):
        """Return the Tx for the given hex string."""
        return class_.parse(io.BytesIO(h2b(hex_string)))

I would like to add support for generating signatures and verifying them against public keys.

Any comments about where to put the new code? I'm thinking of two new methods on Key class.

From vm.py:

...
 # BRAIN DAMAGE -- does it always get down here for each verify op? I think not
...

I can't get signing of a multisig TX to work properly, it seems to me that the redeemscript is dumped when signing and the resulting txin.script is just the signatures without the redeem script ...

I might just be doing something wrong ...

I took the existing test case and trying to work that one out (I also sent a tiny amount of BTC to that address so we can test live ...)

Adding the following line at the end of test_sign_pay_to_script_multisig also throws an Exception.

print ScriptMultisig.from_script(tx2.txs_in[0].script)

spending real BTC:

    def test_sign_pay_to_script_multisig_live(self):
        N, M = 3, 3
        keys = [Key(secret_exponent=i) for i in range(1, M+2)]
        tx_in = TxIn.coinbase_tx_in(script=b'')
        underlying_script = ScriptMultisig(n=N, sec_keys=[key.sec() for key in keys[:M]]).script()
        address = address_for_pay_to_script(underlying_script)
        self.assertEqual(address, "39qEwuwyb2cAX38MFtrNzvq3KV9hSNov3q")
        script = standard_tx_out_script(address)
        tx_out = TxOut(1000000, script)
        tx1 = Tx(version=1, txs_in=[tx_in], txs_out=[tx_out])

        s = Spendable.from_dict(dict(
            coin_value=500000,
            script_hex=b2h(underlying_script),
            tx_hash_hex=b2h_rev(h2b("efdee0cb54de0aef77660b8766f39b7eab69c7d23ec0409709507750f460e587")),
            tx_out_index=0
        ))

        tx2 = tx_utils.create_tx([s], [address])
        hash160_lookup = build_hash160_lookup(key.secret_exponent() for key in keys[:M])
        p2sh_lookup = build_p2sh_lookup([underlying_script])
        tx2.sign(hash160_lookup=hash160_lookup, p2sh_lookup=p2sh_lookup)
        self.assertEqual(tx2.bad_signature_count(), 0)

        # this throws 'n value invalid'
        # print ScriptMultisig.from_script(tx2.txs_in[0].script)

        print tx2.as_hex()

Calling signature_hash for any input greater than 0 barfs on the hash calculation because prior outputs' coin values are set to -1. Traceback from 0.42:

Traceback (most recent call last):
  ...
  File ".../site-packages/pycoin/tx/Tx.py", line 177, in signature_hash
    return from_bytes_32(tmp_tx.hash(hash_type=hash_type))
  File ".../site-packages/pycoin/tx/Tx.py", line 117, in hash
    self.stream(s)
  File ".../site-packages/pycoin/tx/Tx.py", line 103, in stream
    t.stream(f)
  File ".../site-packages/pycoin/tx/TxOut.py", line 48, in stream
    stream_struct("QS", f, self.coin_value, self.script)
  File ".../site-packages/pycoin/serialize/streamer.py", line 48, in stream_struct
    self.stream_lookup[c](f, v)
  File ".../site-packages/pycoin/serialize/bitcoin_streamer.py", line 43, in <lambda>
    "Q": (lambda f: struct.unpack("<Q", f.read(8))[0], lambda f, v: f.write(struct.pack("<Q", v))),
ValueError: cannot convert negative integer to unsigned

A pull request with a fix and test case is forthcoming.

I use this to generate multisig address from N addresses (N=2 in this scenario).

def generate_multisig_address(keys, N=2, M=2):
    from pycoin.tx.pay_to import address_for_pay_to_script
    underlying_script = ScriptMultisig(n=N, sec_keys=[key["key_pair_as_sec"] for key in keys[:M]]).script()
    address = address_for_pay_to_script(underlying_script) #is it possible to get redeemScript?
    return address

However if we change the order of the keys, it will generate different address:

[u'159F7ik2d5emNovLjHgo4A6J9VD3UhZ9gg', u'1B7dPQkjxoFtzoZ4wSnhN3TTVg5Xtiwm8W']
Multisig address: 3MvymRh5nnvqs7vVHivzNBVV5hE9R8FGyt

[u'1B7dPQkjxoFtzoZ4wSnhN3TTVg5Xtiwm8W', u'159F7ik2d5emNovLjHgo4A6J9VD3UhZ9gg']
Multisig address: 3NqzcdAnkQ69wB2d98nJ66AGq3EQEHawyJ

Please consider adding BIP0038 support:

Here is a reference implementation in Python jgarzik/python-bitcoinlib#5

which is specifically this file adding the library: https://github.com/zimage/python-bitcoinlib/blob/e9506aba6d53ad5bf760d67815d711a1c55b2557/bitcoin/bip38.py

If you generate a bunch of private keys using something like genwallet.py and save them (in cold storage), it's harder than it could be to later retrieve them for use with pycoin:

from pycoin.wallet import Wallet
from pycoin.scripts.genwallet import dev_random_entropy

wallet=Wallet.from_master_secret(bytes(bytearray(dev_random_entropy())), is_test=False)
print ('Bitcoin address to receive payments on: %s' % wallet.bitcoin_address())
print ('WIF to store in cold storage: %s' % wallet.wif())

If I later want to look something up about that WIF, how would I create a Wallet object from it? For example, let's say I have a bunch of WIFs but I've lost their corresponding bitcoin addresses. If I could generate a Wallet object from a WIF file, then it'd be as easy as calling bitcoin_address() to determine which WIF goes with which bitcoin address. Does that make sense?

Or, let's say I want to convert from WIF to secret_exponent. I could do this using wif_to_secret_exponent, but having to read the internals and learn about pycoin.encoding.wif_to_secret_exponent is a lot messier vs just having a Wallet object that I can always use that does everything for me easily. I think this would make pycoin much more useful and encourage more developers to use pycoin... Does that make sense?

Apologies if I'm mis-understanding something about how you've structured all these files. I think the library you've built is great!