Git Product home page Git Product logo

bitcoind's Introduction

MIT license Crates Docs

Bitcoind

Utility to run a regtest bitcoind process, useful in integration testing environment.

When the auto-download feature is selected by activating one of the version feature, such as 25_1 for bitcoin core 25.1, starting a regtest node is as simple as that:

// the download feature is enabled whenever a specific version is enabled, for example `25_1` or `24_0_1`
#[cfg(feature = "download")]
{
  use bitcoincore_rpc::RpcApi;
  let bitcoind = bitcoind::BitcoinD::from_downloaded().unwrap();
  assert_eq!(0, bitcoind.client.get_blockchain_info().unwrap().blocks);
}

The build script will automatically download the bitcoin core version 25.1 from bitcoin core, verify the hashes and place it in the build directory for this crate. If you wish to download from an alternate location, for example locally for CI, use the BITCOIND_DOWNLOAD_ENDPOINT env var.

When you don't use the auto-download feature you have the following options:

  • have bitcoind executable in the PATH
  • provide the bitcoind executable via the BITCOIND_EXE env var
use bitcoincore_rpc::RpcApi;
if let Ok(exe_path) = bitcoind::exe_path() {
  let bitcoind = bitcoind::BitcoinD::new(exe_path).unwrap();
  assert_eq!(0, bitcoind.client.get_blockchain_info().unwrap().blocks);
}

Startup options could be configured via the [Conf] struct using [BitcoinD::with_conf] or [BitcoinD::from_downloaded_with_conf]

Issues with traditional approach

I used integration testing based on external bash script launching needed external processes, there are many issues with this approach like:

  • External script may interfere with local development environment 1
  • Use of a single huge test to test everything 2
  • If test are separated, a failing test may fail to leave a clean situation, causing other test to fail (because of the initial situation, not a real failure)
  • bash script are hard, especially support different OS and versions

Features

  • It waits until bitcoind daemon become ready to accept RPC commands
  • bitcoind use a temporary directory as datadir. You can specify the root of your temp directories so that you have node's datadir in a RAM disk (eg /dev/shm)
  • Free ports are asked to the OS. Since you can't reserve the given port, a low probability race condition is still possible, for this reason the process is tried to be spawn 3 times with different ports.
  • The process is killed when the struct goes out of scope no matter how the test finishes
  • Allows easy spawning of dependent processes like:

Thanks to these features every #[test] could easily run isolated with its own environment.

Doc

To build docs:

RUSTDOCFLAGS="--cfg docsrs" cargo +nightly doc --features download,doc --open

MSRV

The MSRV is 1.56.1 for version 0.35.*

Note: to respect 1.56.1 MSRV you need to use and older version of some dependencies, in CI the below dependency versions are pinned:

cargo update
cargo update -p tempfile --precise 3.3.0
cargo update -p log --precise 0.4.18

Pinning in Cargo.toml is avoided because it could cause compilation issues downstream.

Nix

For reproducibility reasons, Nix build scripts cannot hit the internet, but the auto-download feature does exactly that. To successfully build under Nix the user must provide the tarball locally and specify its location via the BITCOIND_TARBALL_FILE env var.

Another option is to specify the BITCOIND_SKIP_DOWNLOAD env var and provide the executable via the PATH.

Alternatively, use the dep without auto-download feature.

Used by

Via bdk dependency

Via electrsd dependency:

bitcoind's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

bitcoind's Issues

Add `exe_path` method

As something like

fn exe_path() -> Option<String> {
        if let Some(downloaded_exe_path) = bitcoind::downloaded_exe_path() {
            downloaded_exe_path
        } else {
            env::var("BITCOIND_EXE").expect(
                "when no version feature is specified, you must specify BITCOIND_EXE env var",
            )
        }
    }

error messages contain no context

When running cargo test on elements-miniscript from a non-master worktree I get

---- test_setup stdout ----
thread 'test_setup' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }', tests/setup/mod.rs:23:86
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

the relevant line is

    if validate_pegin {
        let bitcoind_exe = bitcoind::exe_path().unwrap();
        let bitcoind_conf = bitcoind::Conf::default();
        bitcoind = Some(bitcoind::BitcoinD::with_conf(&bitcoind_exe, &bitcoind_conf).unwrap()); // this unwrap is the error
    }   

There is nothing I can do with this. The test binary does some forking crap so I can't strace it.

v27.1 is NOT 1.41.1 MSRV because of ureq

ureq = "1.0" allows ureq 1.6 to be installed. 1.6 depends on once_cell v1.16.0 which is not msrv 1.41.1. once_cell 1.9.0 is te latest version of at msrv 1.41.1. once_cell 1.10.0 breaks it.

Unfortunately, ureq does not adhere to any MSRV, and only specifies once_cell = "1"

it seems this crate only has 1 ureq call, here https://github.com/RCasatta/bitcoind/blob/2c1758a2d5d638a388906c97bc011802b12e498b/build.rs#L106

so maybe the whole dependency could be replaced with something that maintains an MSRV

This is causing problems downstream. Kixunil/payjoin#36

`cargo test` failing on macOS

  • macOS version: Monterey 12.4
  • Rust version: tried both 1.56 and 1.63

cargo test is failing on macOS with the following error message:

Error msg
running 8 tests
test test::test_local_ip ... ok
test test::test_bitcoind_rpcuser_and_rpcpassword ... ok
test test::test_bitcoind_rpcauth ... FAILED
test test::test_multi_p2p ... FAILED
test test::test_p2p ... FAILED
test test::test_bitcoind ... FAILED
test test::test_data_persistence ... FAILED
test test::test_multi_wallet ... FAILED
 
failures:
 
---- test::test_bitcoind_rpcauth stdout ----
thread 'test::test_bitcoind_rpcauth' panicked at 'called `Result::unwrap()` on an `Err` value: JsonRpc(Rpc(RpcError { code: -4, message: "Wallet file verification failed. SQLiteDatabase: Unable to obtain an exclusive lock on the database, is it being used by another instance of Bitcoin Core?\n", data: None }))', src/lib.rs:683:56
 
---- test::test_multi_p2p stdout ----
thread 'test::test_multi_p2p' panicked at 'called `Result::unwrap()` on an `Err` value: JsonRpc(Rpc(RpcError { code: -4, message: "Wallet file verification failed. SQLiteDatabase: Unable to obtain an exclusive lock on the database, is it being used by another instance of Bitcoin Core?\n", data: None }))', src/lib.rs:573:75
 
---- test::test_p2p stdout ----
thread 'test::test_p2p' panicked at 'called `Result::unwrap()` on an `Err` value: JsonRpc(Rpc(RpcError { code: -4, message: "Wallet file verification failed. SQLiteDatabase: Unable to obtain an exclusive lock on the database, is it being used by another instance of Bitcoin Core?\n", data: None }))', src/lib.rs:524:57
 
---- test::test_bitcoind stdout ----
thread 'test::test_bitcoind' panicked at 'called `Result::unwrap()` on an `Err` value: JsonRpc(Rpc(RpcError { code: -4, message: "Wallet file verification failed. SQLiteDatabase: Unable to obtain an exclusive lock on the database, is it being used by another instance of Bitcoin Core?\n", data: None }))', src/lib.rs:492:43
 
---- test::test_data_persistence stdout ----
thread 'test::test_data_persistence' panicked at 'called `Result::unwrap()` on an `Err` value: JsonRpc(Rpc(RpcError { code: -4, message: "Wallet file verification failed. SQLiteDatabase: Unable to obtain an exclusive lock on the database, is it being used by another instance of Bitcoin Core?\n", data: None }))', src/lib.rs:544:72
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
 
---- test::test_multi_wallet stdout ----
thread 'test::test_multi_wallet' panicked at 'called `Result::unwrap()` on an `Err` value: JsonRpc(Rpc(RpcError { code: -4, message: "Wallet file verification failed. SQLiteDatabase: Unable to obtain an exclusive lock on the database, is it being used by another instance of Bitcoin Core?\n", data: None }))', src/lib.rs:601:43
 
 
failures:
    test::test_bitcoind
    test::test_bitcoind_rpcauth
    test::test_data_persistence
    test::test_multi_p2p
    test::test_multi_wallet
    test::test_p2p
 
test result: FAILED. 2 passed; 6 failed; 0 ignored; 0 measured; 0 filtered out; finished in 17.18s

the output is the same if the tests are run with:

  • cargo test --features 22_0;
  • cargo test with the BITCOIND_EXEC env variable pointing to bitcoind 22.0;
  • cargo test with the BITCOIND_EXEC env variable pointing to bitcoind 23.0.

Can't locate downloaded binary after repository is moved

For some reason, after I moved the location of my repository which uses bitcoind in its dev-dependencies, downloaded_exe_path() continues returning the old path. Everything from target/... onwards is correct - I've verified that there is indeed a bitcoind binary in that location - but the path to the repo seems to remain constant even across system restarts.

How to reproduce

Platform: macOS
Rust version: nightly-2022-06-27-x86_64-apple-darwin
bitcoind crate version: 0.26.1

Have this in Cargo.toml:

[dev-dependencies]
bitcoind = { version = "0.26", features = [ "22_0" ] }

Ensure that OUT_DIR is not set in env vars

echo $OUT_DIR

Have this test:

#[cfg(test)]
mod test {
    fn bitcoind_fails() {
        let conf = Conf::default();
        let exe_path = bitcoind::downloaded_exe_path()
            .expect("Didn't specify bitcoind version in feature flags");

        // This prints
        // [node/src/command/test/mod.rs:72] &exe_path = "<path_to_OLD_repo_location>/target/debug/build/bitcoind-65c3b20abafd4893/out/bitcoin/bitcoin-22.0/bin/bitcoind"
        // Aside from the path to the repo, everything else is correct
        dbg!(&exe_path);

        // This causes the code not to compile
        // let out_dir = std::env!("OUT_DIR");
        // This panics
        // let out_dir = std::env::var_os("OUT_DIR").unwrap();

        // This panics when running from new repository location
        let bitcoind = BitcoinD::with_conf(exe_path, &conf)
            .expect("Failed to init bitcoind");
    }
}

Run the test (should pass)

Move the repository, e.g. from $HOME/dev/myrepo to $HOME/dev/bitcoin/myrepo

Rerun the test (should fail)

What I've tried, but which still produce the same error

  • cargo clean
  • rm -rf <path_to_repo>/target
  • rm -rf ~/.cargo/registry
  • rm -rf ~/.cargo/git
  • cargo cache -e
  • Cloning the repo to a new location on the same computer
  • Even restarting my computer

Workarounds which prevent the panic (but which don't solve the problem)

  • Downgrading to 0_21_0 in Cargo.toml: bitcoind = { version = "0.26", features = [ "0_21_0" ] }
  • Specifying bitcoind using a relative path bitcoind = { path = "../../../github/bitcoind", version = "0.26", features = [ "22_0" ] }
  • Moving the repo back to its original location

Any idea what could be going on?

Look in path for bitcoind

Is there any way to provide the bitcoind binary via the PATH env var? Specifically in build.rs. I am trying to package this project with nix, which strips some of the non-deterministic attributes from gzipped files when generating a checksum, therefore I am getting a different hash than the one you specified here. I do have access to bitcoind v22.0 in nix though, and could easily provide a reference to the binary.

Allow to configure alternate mirrors

It would be nice if we could point the crate to alternate mirrors rather than having https://bitcoincore.org/ hard-coded.

Main motivation is that we in LDK recently started seeing CI failures that seem to come from Github CI being unable to connect to https://bitcoincore.org/:

error: failed to run custom build command for `bitcoind v0.28.1`

Caused by:
  process didn't exit successfully: `/home/runner/work/rust-lightning/rust-lightning/target/debug/build/bitcoind-db625c734508c201/build-script-build` (exit status: 101)
  --- stdout
  filename:bitcoin-23.0-x86_64-linux-gnu.tar.gz version:23.0 hash:2cca490c1f2842884a3c5b0606f179f9f937177da4eadd628e3f7fd7e25d26d0
  url:https://bitcoincore.org/bin/bitcoin-core-23.0/bitcoin-23.0-x86_64-linux-gnu.tar.gz

  --- stderr
  thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Transport(Transport { kind: Io, message: None, url: Some(Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("bitcoincore.org")), port: None, path: "/bin/bitcoin-core-23.0/bitcoin-23.0-x86_64-linux-gnu.tar.gz", query: None, fragment: None }), source: Some(Custom { kind: TimedOut, error: "timed out reading response" }) })', /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/bitcoind-0.28.1/build.rs:106:43

If these issues continue to appear we'd be interested in exploring hosting our own mirror to make the bitcoind/electrsd based tests more robust.

Remove bitcoin core minor versions?

There are 12 Bitcoin core versions available with auto-download features.
To clean up a little bit and save CI jobs I was thinking to remove minor versions if someone doesn't object on this issue in the next months.

Versions removed would be:

  • 0.18.0
  • 0.19.0.1
  • 0.20.0
  • 0.21.0

feature gated `Bitcoind::from_downloaded()`

let bitcoind = bitcoind::BitcoinD::from_donwloaded()).unwrap();

equivalent but slightly more convenient than:

let bitcoind = bitcoind::BitcoinD::new(bitcoind::downloaded_exe_path().unwrap()).unwrap();

Bitcoin file string change in version 23.0

In 23.0 on macos, the bitcoin file string changed from bitcoin-{version}-osx64.tar.gz to bitcoin-{version}-x86_64-apple-darwin.tar.gz, which causes the build script to error out when using version 23.0

Remove the `time = "=0.3.10"` hack once MSRV reaches 1.59

I'd just like to request that the time = "=0.3.10" hack is removed once bitcoind's MSRV reaches 1.59. I have confirmed that the bitcoind build succeeds with time = "0.3" if compiling with version 1.59.

More information

The latest 0.28.1 release which introduced the time = "=0.3.10" hack to support MSRV 1.57 broke our build, which contains serde_with.

Steps to reproduce

cargo new --lib repro-build

Cargo.toml:

[package]
name = "repro-build"
version = "0.1.0"
edition = "2021"

[dependencies]
serde_with = "=2.1.0"
bitcoind = "=0.28.1"
$ cargo build

error: failed to select a version for `time`.
    ... required by package `serde_with v2.1.0`
    ... which satisfies dependency `serde_with = "=2.1.0"` of package `repro-build v0.1.0 (/Users/fang/temp/repro-build)`
versions that meet the requirements `~0.3.11` are: 0.3.17, 0.3.16, 0.3.15, 0.3.14, 0.3.13, 0.3.12, 0.3.11

all possible versions conflict with previously selected packages.

  previously selected package `time v0.3.10`
    ... which satisfies dependency `time = "=0.3.10"` of package `bitcoind v0.28.1`
    ... which satisfies dependency `bitcoind = "=0.28.1"` of package `repro-build v0.1.0 (/Users/fang/temp/repro-build)`

failed to select a version for `time` which could resolve this conflict

Solution

Downgrading our serde_with caused us other problems (specifically, serde_with 2.0.0 is broken), so we had to downgrade bitcoind to 0.28.0 to fix our build. We would like to switch back to the latest version of bitcoind at some point, but to do so requires removing the time = "=0.3.10" hack currently in the Cargo.toml.

Possible workaround to remove the hack while keeping MSRV at 1.57

I spent some time playing around with bitcoind and rust 1.57 and was able to reproduce the 1.57 build problem. Interestingly, after building the crate once with time = "=0.3.10" and generating the Cargo.lock, cargo build still succeeds if you change the dependency back to time = "0.3". If there is a way to specify that time should be version 0.3.10 in the cargo invocation, bitcoind's CI can be fixed, then the hack can be removed, allowing libraries and binaries downstream of bitcoind to avoid this dependency resolution problem. In other words, the CI script seems a better place to put a hack than the Cargo.toml, which has ripple effects on downstream crates.

Flakyness in tests

Some times CI reports this error and usually a re-kick solve the issue, find the origin of the flakyness

---- test::test_multi_wallet stdout ----
thread 'test::test_multi_wallet' panicked at 'assertion failed: (left == right)
left: Amount(1.00000000 BTC),
right: Amount(0.00000000 BTC)', src/lib.rs:635:9

Use the `home` crate to determine $CARGO_HOME

It looks like the recommended way to get the $CARGO_HOME value is from the home crate:

https://crates.io/crates/home

Also @ulrichard ran into an error that could be related (he's on a Debian Bullseye system) and got this error:

When I execute "cargo test" on the current master, I get a compiler error:
Compiling bitcoincore-rpc v0.13.0
error: failed to run custom build command for bitcoind v0.13.0

Caused by:
  process didn't exit successfully: /home/richi/src/github/bdk/target/debug/build/bitcoind-4c60e35c276999bb/build-script-build (exit status: 101)
  --- stderr
  thread 'main' panicked at 'called Result::unwrap() on an Err value: NotPresent', /home/richi/.cargo/registry/src/github.com-1ecc6299db9ec823/bitcoind-0.13.0/build.rs:39:78
  note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed

it was fixed when he manually set his $CARGO_HOME env variable.

Doesn't work with Alpine linux

The binary that this crate downloads from bitcoincore.org doesn't run on an Alpine Docker container with
FROM alpine:3.15

The shared library ld-linux-x86-64.so.2 is missing and no compatible version available from the Alpine repo:
`/app # /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind --version
sh: /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: not found

/app # ldd /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind
/lib64/ld-linux-x86-64.so.2 (0x7f53f39aa000)
libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7f53f39aa000)
librt.so.1 => /lib64/ld-linux-x86-64.so.2 (0x7f53f39aa000)
libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f53f39aa000)
libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x7f53f2d2e000)
libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f53f39aa000)
Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind)
Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: __strdup: symbol not found
Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: __rawmemchr: symbol not found
Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: __snprintf_chk: symbol not found
Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: __fread_chk: symbol not found
Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: __strftime_l: symbol not found
Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: __vsnprintf_chk: symbol not found
Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: pthread_yield: symbol not found
Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: malloc_info: symbol not found
Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: __fprintf_chk: symbol not found
Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: __memcpy_chk: symbol not found
`

Also using the packaged version of bitcoind from the alpine repository doesn't work, because it was compiled without wallet functionality.
https://git.alpinelinux.org/aports/tree/community/bitcoin/APKBUILD line 51

I'm not sure what is the best way to make it work with Alpine.

Node connection not working for multiple nodes

I am hitting this weird issue where if I spawn 3 nodes and try to make the following connection between them

Node1 <--- Node2 <--- Node3 (<--- : "connects to")

I get the connection between Node1 and Node2 but not between Node2 and Node3.

It can be reproduced by the following test

// Create Node 1
        let mut conf_node1 = Conf::default();
        conf_node1.p2p = P2P::Yes;
        let node1 =
            BitcoinD::with_conf(bitcoind::downloaded_exe_path().unwrap(), &conf_node1).unwrap();

        // Create Node 2 connected Node 1
        let node1_connection = node1.p2p_connect().unwrap();
        let mut conf_node2 = Conf::default();
        conf_node2.p2p = node1_connection;
        let node2 =
            BitcoinD::with_conf(bitcoind::downloaded_exe_path().unwrap(), &conf_node2).unwrap();

        // Create Node 3 Connected To Node 2
        let node2_connection = node2.p2p_connect().unwrap();
        let mut conf_node3 = Conf::default();
        conf_node3.p2p = node2_connection;
        let node3 =
            BitcoinD::with_conf(bitcoind::downloaded_exe_path().unwrap(), &conf_node3).unwrap();

        // Get each nodes Peers
        let node1_peers = node1.client.get_peer_info().unwrap();
        let node2_peers = node2.client.get_peer_info().unwrap();
        let node3_peers = node3.client.get_peer_info().unwrap();

        // Peers found for Node 1 and 2, but not for Node 3
        assert_eq!(node1_peers.len(), 1);
        assert_eq!(node2_peers.len(), 1);
        assert_eq!(node3_peers.len(), 0);

Address the race condition

If my understanding is correct this crate launches a separate instance per test, which is nice but because of the race condition and tests being run in parallel this can cause flaky tests which is quite bad. It'd be great to fix the race completely.

I took a look at the obvious "what happens if I set -port 0 -rpcport 0" The answer is `bitcoind seems to attempt to bind the default port anyway (WTF?) and doesn't log the actual port so if we do this we have no way of figuring it out.

I see only two solutions that include existing bitcoind instances:

  1. On platforms that support it we could LD_PRELOAD a library that overrides bind() to pass in port 0 and then reports the mapping over a pipe. We could read that report and learn the actual port number.
  2. Create private network namespace which has a custom IP and is somehow allowed to communicate with the parent. This should be possible on all platforms which support Docker because it presumably does this but it may require root which is not nice.

Of course the long term solution is to add direct support to bitcoind itself.

Should allow both env var and feature

Right now if a feature gate is set then the crate errors out, which means it is impossible for a user to override the feature flag.

Instead if the environment variable is set it should always take precedence.

Making `bitcoind` and `electrsd` production-grade?

The bitcoind / electrsd crates have been enormously helpful for our integration testing, and we (Lexe) think they could be very useful in production as well.

Our interest in a production-grade bitcoind/electrs Rust harnesses: Due to the headaches of managing bash scripts / config files / systemd service files and whatnot, we actually decided to write our production (as well as testnet, dev, regtest etc) orchestration code in pure Rust, so that we could handle all of our argument shuttling in a strongly-typed language, resulting in a sharp reduction in deployment errors. Cleaning up bitcoind and electrsd to be "production-grade" software would allow us to extend our current paradigm to the deployment of these services as well, and allows bitcoind / electrsd arguments to be programmatically configurable in production. We also have an internal framework which centralizes things like CLI arguments, config files, and environment variables, so exposing Rust hooks into a bitcoind/electrs harness allows us to reuse this system as well.

So, I wanted to start a discussion on this. Some questions:

  • Is anyone else interested in this?
  • What is the current readiness of the bitcoind / electrsd crates from a "production" standpoint, and what work would need to be done to accomplish this?
  • Is it better to pursue this development goal under this repository, or should interested parties add these capabilities in a permanent fork which does not have the intention of being merged back upstream? (Similar to the romanz/Blockstream electrsd fork situation). In other words, are the requirements of a "production-ready" vs integration test-oriented bitcoind/electrs harness so different that it would be better to develop them in separate repositories? (My hunch is this is not so).

Especially interested in your thoughts @RCasatta.

Download using http proxy

It would be really cool, if downloading the executables would use a proxy if one of the following environmant variables are set:

  • HTTP_PROXY
  • HTTPS_PROXY

Separate client connection parameter, not passing bitcoind to other process

electrsd is launched like

let electrs = ElectrsD::new(electrs_exe, bitcoind, true, false).unwrap();

but it may be useful to start another electrs instance or other bitcoin rpc client, something like

let electrs1 = ElectrsD::new(electrs_exe, bitcoind.params(), true, false).unwrap();
let electrs2 = ElectrsD::new(electrs_exe, bitcoind.params(), true, false).unwrap();

Change: `args: Vec<&str>`

using Vec<&str> as args in Conf doesn't allow the caller to use String, use something allowing that (T: Borrow<&str>, Vec<T>?)

Cannot use `-rpcuser` and `-rpcpassword` as arguments, does not return Err and enters infinite loop

Describe the bug
Cannot use -rpcuser=<user> and -rpcpassword=<password> args with bitcoind, although it's indicated to be deprecated when running bitcoind from the binaries I think it's useful and would be great to maintain compatibility.

The -rpcuser and -rpcpassword does not create a .cookie file, therefore it produces an error when trying to read from the file and get auth parameters.

It enters in an infinite loop because it cannot create and connect a RPC client in step: let client_result = Client::new(&rpc_url, Auth::CookieFile(cookie_file.clone())); when calling auth.get_user_pass it returns an io error because the file does not exists, but the error is not treated and it skip to another pass in the loop considering it's just because the process is not up yet.

It would be better to not consider the .cookie file when receiving -rpcuser and -rpcpassword arguments or raise an error recommending to use -rpcauth, which works because it uses user and password and creates a .cookie file, and I'm using as a workaround.

To Reproduce
You can simply try to start a process passing the -rpcauth and -rpcpassword as arguments in conf, it won't event start the process.
I've created two tests here: oleonardolima@b522d5e

Expected behavior
I expect that it would work successfully or return an error indicating that it's indicated to use -rpcauth instead.

Build environment

  • bitcoind tag/commit: v0.13.0 | bd952c186ffa30e328a7b12c350569856b43c7c1
  • OS+version: macOS 12.4
  • Rust/Cargo version: rustc 1.60.0 (7737e0b5c 2022-04-04) | cargo 1.60.0 (d1fd9fe2c 2022-03-01)
  • Rust/Cargo target: aarch64-apple-darwin

Windows support

Looks like the build script is missing support for windows platforms. Assuming that there just wasn't any need for this and that there's not some fundamental problem. From my limited look at the codebase seems like we just need to implement a download_filename implementation for windows?

Fails to compile on new macs?

A user reported this on one of the new m1 macs:

error[E0277]: the size for values of type `str` cannot be known at compilation time
--> /Users/s/.cargo/registry/src/github.com-1ecc6299db9ec823/bitcoind-0.17.0/build.rs:50:9
|
50 | / println!(
51 | | "filename:{} version:{} hash:{}",
52 | | download_filename, VERSION, expected_hash
53 | | );
| |__________^ doesn't have a size known at compile-time
|
= help: the trait `Sized` is not implemented for `str`
= note: this error originates in the macro `$crate::format_args_nl` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0277]: the size for values of type `str` cannot be known at compilation time
--> /Users/s/.cargo/registry/src/github.com-1ecc6299db9ec823/bitcoind-0.17.0/build.rs:55:19
|
55 | let url = format!(
| ___________________^
56 | | "https://bitcoincore.org/bin/bitcoin-core-{}/{}",
57 | | VERSION, download_filename
58 | | );
| |_________^ doesn't have a size known at compile-time
|
= help: the trait `Sized` is not implemented for `str`
= note: this error originates in the macro `$crate::__export::format_args` (in Nightly builds, run with -Z macro-backtrace for more info)

error: aborting due to 3 previous errors

Some errors have detailed explanations: E0277, E0425.
For more information about an error, try `rustc --explain E0277`.
error: could not compile `bitcoind`

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed

Can anyone confirm or deny?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.