lu-zero / autotools-rs Goto Github PK

The Config::build() method automatically emits the cargo directive cargo:root=. Please provide a method to disable this behavior, e.g., Config::emit_cargo_root(bool).

The default Config instance would emit cargo:root= to make this change backwards-compatible, and a call to config.emit_cargo_root(false) would avoid emitting the directive.

Right now autotools-rs builds into OUT_DIR but also configures the installation prefix as OUT_DIR. That means the build files get mixed in with the installation artifacts. Not generally a huge deal, but I have a use case where I want to extract the compiled artifacts without the build artifacts downstream of Cargo.

This is my current workaround:

https://github.com/MaterializeInc/rust-protobuf-native/blob/cbc2b33ac0d474ca557018f4695cc508076ffeb0/protobuf-src/build.rs#L22-L32

Would you be open to a patch to bring that behavior upstream? If so, would you prefer it to be configurable or just to make this the default behavior? (FWIW I can't think of a reason that someone would prefer the current behavior to separate build and install directories.)

I am replacing this code in my build.rs (which works fine):

    let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap());
    
    Command::new("./configure")
        .current_dir("ta-lib")
        .arg(format!("--prefix={}", out_dir.display()))
        .output()
        .expect("Failed to execute TA C library configure script");

    Command::new("make")
        .current_dir("ta-lib")
        .arg("install")
        .output()
        .expect("Failed to build TA C library");

    println!(
        "cargo:rustc-link-search={}",
        out_dir.join("lib").display()
    );
    println!("cargo:rustc-link-lib=ta_lib");

With this:

    let dst = autotools::build("ta-lib");

    println!("cargo:rustc-link-search=native={}", dst.display());
    println!("cargo:rustc-link-lib=static=ta-lib");

Which fails with several headers not being found, e.g.:

In file included from /Users/moritz/code/crates/ta-lib-rs/ta-lib-sys/ta-lib/src/ta_abstract/ta_group_idx.c:48:
  /Users/moritz/code/crates/ta-lib-rs/ta-lib-sys/ta-lib/src/ta_abstract/ta_def_ui.h:44:13: fatal error: 'ta_abstract.h' file not found
     #include "ta_abstract.h"

What am I missing?

This functionality would be useful when some rust code only requires headers to be available that are generated via running the configure script, e.g. config.h, and doesn't need to build the target via make.

From glancing through the code it doesn't look like the functionality is split in this fashion yet so one can only run the configure script while also running a pointless build via make.

Hey,

I tried running this on Windows (disclaimer I'm not a Windows guy).
Installed mingw-gcc, mingw-make, mingw-autotools but autoreconf is not an executable on Windows (it is an perl file) basically it doesn't find it in the path because it requires perl autoreconf.

A potential workaround is to add typealias but I prefer not to that

Am I doing something wrong? Alternatively shall it be $ perl autoreconf on Windows?

Thanks

Hi @lu-zero, any chance you could cut a new release with the latest commits on master?

Is there a way to reconfigure?

I see there's a reconfig member but there's no way to set it. Documentation says to call reconf() but there's no such method.

I'm using this crate to vendor and build dependencies for my spatialite-sys crate. Unfortunately, the hard-coded --disable-shared --enable-static options cause failures in Spatialite's configure script. I guess it searches for some of its dependent functions and expects to find them in a shared library.

I don't know what the best solution is, but for now I removed --disable-shared and everything now works. I assume this means that static libraries still build where supported, but that shared libraries are available for configure checks that need them. Seems like it might be the best of both worlds. I'm happy to contribute a PR to that effect if you think it's the best way forward.

Hey,

According to the docs for enable and disable opt parameter is supposed to not require to begin with a - but the implementation never concatenate with a - https://github.com/lu-zero/autotools-rs/blob/master/src/lib.rs#L348-#L361

So for example:
.enable("foo", None) -> --enablefoo

Sometimes it would be desirable to catch a build error to report a custom error message rather than directly panicking.

maybe it would be possible to do implement a try_build which returns a ->Result<PathBuf,usize>

Write some test to make sure everything works:

• Test options
• Test autoreconf
• Test env overrides

I use simple build.rs:

extern crate autotools;

fn main() {
    let dst = autotools::build("3rdparty/rsync-3.1.3");
    println!(
        "rustc-env=RSYNC_BIN={}",
        dst.join("bin").join("rsync").display()
    );
}

and after adding such build.rs build becomes really slow,
because of it runs configure every build.
It would be nice check if Makefile exists and if so just run make.
As I know make automatically runs configure if something changed.

The liblzma backdoor (CVE-2024-3094) took advantage of autotools using arcane language generating unreadable gibberish to hide injection of malware payload.

OTOH lzma-sys crate uses a custom build.rs instead of autotools, which is simpler and easier to audit than the upstream configure.ac, not just configure.

This makes me wonder whether this crate could do something to mitigate the risk of using autotools. Perhaps the whole crate could be deprecated, to raise awareness of supply chain risks of autotools, and encourage Rust sys crates to migrate to the cc crate, OS packages, or some modern build systems instead?

Thanks for this! Looks like it has the potential to help solve some annoying problems with cross-platform builds for one of my crates.

I notice that the .cflags(...) method doesn't seem to do anything, other than push to a variable that is never referenced again.

I'm working on a fairly complex build that needs lots of native libs. Essentially what I need to do is set -I out_dir/include so dependent libraries' headers can be found. Wondering if that might be a sensible default, or if a helper might be added to set that up?

Thanks again.

I'm trying to make a package that compiles LibreSSL, to be used in a later version or fork of openssl-sys, in place of openssl-src.

The LibreSSL build setup (unlike OpenSSL's bespoke system) uses the standard autotools ./configure; make install. Everything works great for my package when the build/host and target systems coincide. I'm just doing this:

    use autotools::Config;
    let mut cfg = Config::new(&inner_dir);
    cfg.disable_shared();
    cfg.out_dir(&install_dir);
    cfg.cflag("-v");
    let dst = cfg.build();

However, things don't work right when I try to test for systems where host and target come apart. I'm using a ci infrastructure that uses github's Ubuntu machines and Docker instances to test on different architectures. For example, one of the non-working tests is for aarch64-unknown-linux-gnu.

Here is the .github/workflows/main.yml file:

name: CI
on: [push, pull_request]

jobs:
  test:
    name: Test
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        thing:
        - stable
        - beta
        - nightly
        - macos-x86_64
        include:
        - thing: stable
          target: x86_64-unknown-linux-gnu
          rust: stable
          os: ubuntu-18.04
        - thing: beta
          target: x86_64-unknown-linux-gnu
          rust: beta
          os: ubuntu-18.04
        - thing: nightly
          target: x86_64-unknown-linux-gnu
          rust: nightly
          os: ubuntu-18.04
        - thing: macos-x86_64
          target: x86_64-apple-darwin
          rust: stable
          os: macos-10.15
        - thing: aarch64-linux
          target: aarch64-unknown-linux-gnu
          rust: stable
          os: ubuntu-18.04

    steps:
    - uses: actions/checkout@v1
      with:
        submodules: true
    - name: Install Rust (rustup)
      run: rustup update ${{ matrix.rust }} --no-self-update && rustup default ${{ matrix.rust }}
      shell: bash
    - run: rustup target add ${{ matrix.target }}
    - name: Set crt-static
      if: matrix.crt_static == 'yes'
      run: echo ::set-env name=RUSTFLAGS::-Ctarget-feature=+crt-static
      shell: bash
    - run: |
        set -e
        cargo generate-lockfile
        ./ci/run-docker.sh ${{ matrix.target }}
      if: "!startsWith(matrix.os, 'windows')"
      name: Run tests (not Windows)
    - run: |
        cargo test --manifest-path testcrate/Cargo.toml --target ${{ matrix.target }}
        cargo test --manifest-path testcrate/Cargo.toml --target ${{ matrix.target }} --release
        cargo run --release --target ${{ matrix.target }} --manifest-path testcrate/Cargo.toml --features package
      if: startsWith(matrix.os, 'windows')
      name: Run tests (Windows)
      shell: cmd

  rustfmt:
    name: Rustfmt
    runs-on: ubuntu-18.04
    steps:
    - uses: actions/checkout@v1
    - name: Install Rust
      run: rustup update stable && rustup default stable && rustup component add rustfmt
    - run: cargo fmt -- --check

and here is ci/run-docker.sh:

#!/bin/sh

target=$1

if [ ! -d ci/docker/$1 ]; then
  exec ci/run.sh $1
fi

set -ex

docker build \
  --rm \
  --tag libressl-src-ci \
  ci/docker/$1

docker run \
  --rm \
  --volume `rustc --print sysroot`:/rust:ro \
  --volume `pwd`:/usr/code:ro \
  --volume `pwd`/target:/usr/code/target \
  --volume $HOME/.cargo:/cargo \
  --env CARGO_HOME=/cargo \
  --workdir /usr/code \
  libressl-src-ci \
  bash -c "PATH=\$PATH:/rust/bin ci/run.sh $target"

and here ci/run.sh:

target=$1
set -ex
cargo test --manifest-path testcrate/Cargo.toml --target $1 -vv
cargo test --manifest-path testcrate/Cargo.toml --target $1 -vv --release

and here ci/docker/aarch64-unknown-linux-gnu/Dockerfile:

FROM ubuntu:18.04

RUN apt-get update -y && apt-get install -y --no-install-recommends \
  ca-certificates \
  make \
  file \
  gcc \
  libc6-dev \
  gcc-aarch64-linux-gnu \
  libc6-dev-arm64-cross
ENV CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_RUNNER=echo \
  CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc

When running the job for that target, what happens is that my package invokes "/usr/code/target/aarch64-unknown-linux-gnu/debug/build/testcrate-185e7fb7707c88c7/out/libressl-build/build/src/configure" "--host=x86_64-unknown-lin ux-gnu" "--prefix=/usr/code/target/aarch64-unknown-linux-gnu/debug/build/testcrate-185e7fb7707c88c7/out/libressl-build" "--disable-shared" "--enable-static"

And then invokes make install. But the libraries that get built don't seem to be the ones that the linker, aarch64-linux-gnu-gcc, expects. I get errors like this:

  = note: /usr/lib/gcc-cross/aarch64-linux-gnu/7/../../../../aarch64-linux-gnu/bin/ld: skipping incompatible /usr/code/target/aarch64-unknown-linux-gnu/debug/build/testcrate-185e7fb7707c88c7/out/libressl-build/lib/libtls.a when searching for -ltls
          /usr/lib/gcc-cross/aarch64-linux-gnu/7/../../../../aarch64-linux-gnu/bin/ld: cannot find -ltls

Digging into this further, it seems that the ./configure; make install process is never using an aarch64-targetting tool until it gets to the linker, as specified in the env var CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc.

Why is that? At first I thought this may be related to #15, but when I made a fork of this repository that added that --target=$TARGET to the invocation of ./configure, it didn't resolve anything.

Looking into this repository's source code, it's hard to see what mechanism should be used to specify the compiler in a context like this. The pub fn build begins with:

    let mut c_cfg = cc::Build::new();
    c_cfg.cargo_metadata(false)
        .opt_level(0)
        .debug(false)
        .target(&target)
        .warnings(false)
        .host(&host);
    let c_compiler = c_cfg.get_compiler();

And that cc::Tool instance correctly selects my aarch64-targetting compiler, but then that information never gets passed to the invocation of ./configure and/or make.

How am I supposed to specify using this library that the make should not be using the native compiler? Should I be passing env variables like CC to my autotools::Config instance? Or should I be setting other CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_* variables in the same way that the linker was specified?

(I borrowed the GitHub ci infrastructure from another library and am adapting it to my own needs; I guess these questions demonstrate that I don't understand it 100%.)

Initially reported in HadrienG2/hwlocality#91, looks like cross-compilation (at least on macOS) doesn't pass correct flags down the stack.

Here is compilation on x86-64 macOS for aarch64:

  configure: error: in `/Users/runner/work/subspace/subspace/target/aarch64-apple-darwin/production/build/hwlocality-sys-5187208e9a266943/out/build':
  configure: error: cannot run C compiled programs.
  If you meant to cross compile, use `--host'.
  See `config.log' for more details
  thread 'main' panicked at /Users/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/autotools-0.2.6/src/lib.rs:781:5: