Comments (15)
saethlin
commented on June 28, 2024
1
I'm sure you have the error, but I'd like to be able to induce it on my own so that I can minimize and/or debug the issue. I'm decently certain that there is something about your system state that is permitting you to run these commands.
So, I'm up for shooting in the dark here but hopes are not high without the ability to reproduce your build setup. I see you're running via some exec wrapper, which runs export LD_PRELOAD="/usr/lib/libmimalloc.so". What happens if you don't try to override the allocator with LD_PRELOAD?
from rust.
veera-sivarajan
commented on June 28, 2024
@rustbot label -needs-triage +I-crash +regression-from-stable-to-stable
from rust.
mitzsch
commented on June 28, 2024
Same issue over here.
`warning: `/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.cargo/config` is deprecated in favor of `config.toml`
note: if you need to support cargo 1.38 or earlier, you can symlink `config` to `config.toml`
warning: `/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.cargo/config` is deprecated in favor of `config.toml`
note: if you need to support cargo 1.38 or earlier, you can symlink `config` to `config.toml`
warning: /__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/src_packages/rav1e/Cargo.toml: unused manifest key: lints.rust.unexpected_cfgs.check-cfg
Updating crates.io index
Downloading crates ...
Downloaded crossbeam v0.8.4
Downloaded console v0.15.8
Downloaded v_frame v0.3.8
Downloaded clap_builder v4.4.18
Downloaded syn v2.0.66
Downloaded aligned-vec v0.6.0
Downloaded thiserror-impl v1.0.61
Downloaded libc v0.2.155
Downloaded thiserror v1.0.61
Downloaded profiling-procmacros v1.0.15
Downloaded profiling v1.0.15
Downloaded proc-macro2 v1.0.85
Downloaded paste v1.0.15
Downloaded num-derive v0.4.2
Downloaded new_debug_unreachable v1.0.6
Downloaded equator-macro v0.2.1
Downloaded equator v0.2.2
Downloaded clap_complete v4.4.10
Downloaded clap v4.4.18
Downloaded cc v1.0.98
Downloaded built v0.7.3
Compiling libc v0.2.155
Compiling proc-macro2 v1.0.85
Compiling unicode-ident v1.0.12
Compiling autocfg v1.1.0
error: rustc interrupted by SIGSEGV, printing backtrace
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d6f66a8619a171d6.so(+0x2e40fd3)[0x7f9e9ec40fd3]
/usr/lib/libc.so.6(+0x3cae0)[0x7f9e9bc50ae0]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc(+0x23a9e)[0x555dfdcbaa9e]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc(+0x1c427)[0x555dfdcb3427]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.78.0-stable(_ZN4llvm3sys22DontRemoveFileOnSignalENS_9StringRefE+0x1b2)[0x7f9e9a7f0a0e]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.78.0-stable(_ZN4llvm3sys2fs8TempFile4keepERKNS_5TwineE+0x4c)[0x7f9e9a7f114c]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.78.0-stable(_ZN4llvm12writeArchiveENS_9StringRefENS_8ArrayRefINS_16NewArchiveMemberEEENS_17SymtabWritingModeENS_6object7Archive4KindEbbSt10unique_ptrINS_12MemoryBufferESt14default_deleteIS9_EEb+0x21b)[0x7f9e9a7f109b]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d6f66a8619a171d6.so(+0x5174d8c)[0x7f9ea0f74d8c]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d6f66a8619a171d6.so(_RNvXs_NtNtCskFMTEvEfYcy_18rustc_codegen_llvm4back7archiveNtB4_18LlvmArchiveBuilderNtNtNtCskzQGxkPzHAl_17rustc_codegen_ssa4back7archive14ArchiveBuilder5build+0x32c)[0x7f9ea0bc112c]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d6f66a8619a171d6.so(_RNvNtNtCskzQGxkPzHAl_17rustc_codegen_ssa4back4link11link_binary+0x9b7)[0x7f9ea0bbdc05]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d6f66a8619a171d6.so(_RNvXs5_CskFMTEvEfYcy_18rustc_codegen_llvmNtB5_18LlvmCodegenBackendNtNtNtCskzQGxkPzHAl_17rustc_codegen_ssa6traits7backend14CodegenBackend4link+0x24)[0x7f9ea0bbd224]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d6f66a8619a171d6.so(_RNvMs4_NtCsfYYydOQbPZl_15rustc_interface7queriesNtB5_6Linker4link+0x307)[0x7f9ea0cf0d97]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d6f66a8619a171d6.so(+0x4bfcd63)[0x7f9ea09fcd63]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d6f66a8619a171d6.so(+0x4f3eb6e)[0x7f9ea0d3eb6e]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d6f66a8619a171d6.so(+0x4f3e9ca)[0x7f9ea0d3e9ca]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libstd-d2ef02247056996e.so(rust_metadata_std_e4dfbc2c3f4b09f1+0xc1145)[0x7f9ea1dd8145]
/usr/lib/libc.so.6(+0x92ded)[0x7f9e9bca6ded]
/usr/lib/libc.so.6(__clone+0x44)[0x7f9e9bd29ee4]
note: we would appreciate a report at https://github.com/rust-lang/rust
error: could not compile `unicode-ident` (lib)
Caused by:
process didn't exit successfully: `/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc --crate-name unicode_ident --edition=2018 /__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.cargo/registry/src/index.crates.io-6f17d22bba15001f/unicode-ident-1.0.12/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C metadata=1846433f2535c8aa -C extra-filename=-1846433f2535c8aa --out-dir /__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/build64-v3/packages/rav1e-prefix/src/rav1e-build/release/deps -C strip=symbols -L dependency=/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/build64-v3/packages/rav1e-prefix/src/rav1e-build/release/deps --cap-lints allow` (signal: 11, SIGSEGV: invalid memory reference)
warning: build failed, waiting for other jobs to finish...
error: rustc interrupted by SIGSEGV, printing backtrace
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d6f66a8619a171d6.so(+0x2e40fd3)[0x7f0778e40fd3]
/usr/lib/libc.so.6(+0x3cae0)[0x7f0775d28ae0]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc(+0x23a9e)[0x556c9bacaa9e]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc(+0x1c427)[0x556c9bac3427]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.78.0-stable(_ZN4llvm3sys22DontRemoveFileOnSignalENS_9StringRefE+0x1b2)[0x7f07747f0a0e]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.78.0-stable(_ZN4llvm3sys2fs8TempFile4keepERKNS_5TwineE+0x4c)[0x7f07747f114c]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.78.0-stable(_ZN4llvm12writeArchiveENS_9StringRefENS_8ArrayRefINS_16NewArchiveMemberEEENS_17SymtabWritingModeENS_6object7Archive4KindEbbSt10unique_ptrINS_12MemoryBufferESt14default_deleteIS9_EEb+0x21b)[0x7f07747f109b]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d6f66a8619a171d6.so(+0x5174d8c)[0x7f077b174d8c]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d6f66a8619a171d6.so(_RNvXs_NtNtCskFMTEvEfYcy_18rustc_codegen_llvm4back7archiveNtB4_18LlvmArchiveBuilderNtNtNtCskzQGxkPzHAl_17rustc_codegen_ssa4back7archive14ArchiveBuilder5build+0x32c)[0x7f077adc112c]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d6f66a8619a171d6.so(_RNvNtNtCskzQGxkPzHAl_17rustc_codegen_ssa4back4link11link_binary+0x9b7)[0x7f077adbdc05]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d6f66a8619a171d6.so(_RNvXs5_CskFMTEvEfYcy_18rustc_codegen_llvmNtB5_18LlvmCodegenBackendNtNtNtCskzQGxkPzHAl_17rustc_codegen_ssa6traits7backend14CodegenBackend4link+0x24)[0x7f077adbd224]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d6f66a8619a171d6.so(_RNvMs4_NtCsfYYydOQbPZl_15rustc_interface7queriesNtB5_6Linker4link+0x307)[0x7f077aef0d97]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d6f66a8619a171d6.so(+0x4bfcd63)[0x7f077abfcd63]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d6f66a8619a171d6.so(+0x4f3eb6e)[0x7f077af3eb6e]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d6f66a8619a171d6.so(+0x4f3e9ca)[0x7f077af3e9ca]
/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libstd-d2ef02247056996e.so(rust_metadata_std_e4dfbc2c3f4b09f1+0xc1145)[0x7f0775f99145]
/usr/lib/libc.so.6(+0x92ded)[0x7f0775d7eded]
/usr/lib/libc.so.6(__clone+0x44)[0x7f0775e01ee4]
note: we would appreciate a report at https://github.com/rust-lang/rust
error: could not compile `autocfg` (lib)
Caused by:
process didn't exit successfully: `/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc --crate-name autocfg --edition=2015 /__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/install_rustup/.cargo/registry/src/index.crates.io-6f17d22bba15001f/autocfg-1.1.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C metadata=4f9cf7857f0121b7 -C extra-filename=-4f9cf7857f0121b7 --out-dir /__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/build64-v3/packages/rav1e-prefix/src/rav1e-build/release/deps -C strip=symbols -L dependency=/__w/mpv-winbuild/mpv-winbuild/mpv-winbuild-cmake/build64-v3/packages/rav1e-prefix/src/rav1e-build/release/deps --cap-lints allow` (signal: 11, SIGSEGV: invalid memory reference)
Error: CliError { error: Some(2 jobs failed), exit_code: 101 }
from rust.
saethlin
commented on June 28, 2024
I am also on Arch. cargo install cargo-c --profile=release-strip --features=vendored-openssl does not crash.
I also see that you are both reporting output that contains mention of a directory called install_rustup, but I do not see such a directory in the repo. Perhaps it is generated by the build process? I suspect that whatever settings the Cargo config in there contains is required to reproduce the crash.
I'm trying to follow the directions in this repo. Unfortunately it seems like there's no sandboxing in sight, so I'm going to try to do this all in an archlinux:latest Docker image. I start up the image and run pacman -Syyuu, then try to follow the directions.
[root@f3b6f253925d ~]# pacman -S git gyp mercurial subversion ninja cmake meson ragel yasm nasm asciidoc enca gperf unzip p7zip gcc-multilib clang lld libc++ libc++abi python-pip curl lib32-gcc-libs lib32-glib2 mimalloc ccache
warning: curl-8.8.0-1 is up to date -- reinstalling
error: target not found: lib32-glib2
So I just deleted that package from the list.
Then the setup instructions tell me to run pip3, but that also doesn't work on Arch. So I created a virtual environment and installed the Python packages in there.
Then I skip down to the "Compiling with GCC" section, and I try to paste in the cmake command. I suppose I'm supposed to clone the repo first? The cmake command reports this, after I clone the repo:
(venv) [root@f3b6f253925d ~]# cmake -DTARGET_ARCH=x86_64-w64-mingw32 -DGCC_ARCH=x86-64-v3 -DSINGLE_SOURCE_LOCATION="/home/user/packages" -DRUSTUP_LOCATION="/home/user/install_rustup" -G Ninja -B build64 -S mpv-winbuild-cmake
-- MINGW_INSTALL_PREFIX dir: /root/build64/install/x86_64-w64-mingw32
-- SINGLE_SOURCE_LOCATION dir: /home/user/packages
-- RUSTUP_LOCATION dir: /home/user/install_rustup
-- Compiler used: gcc
-- Malloc: /usr/lib/libmimalloc.so
-- Ccache: OFF
-- Target toolchain to build: x86_64-w64-mingw32
CMake Error at cmake/custom_steps.cmake:130 (if):
if given arguments:
"EXISTS" "/home/user/packages/mingw-w64" "GIT_CLONE_FLAGS" "--filter=tree" "0/.git"
Unknown arguments specified
Call Stack (most recent call first):
toolchain/mingw-w64.cmake:12 (force_rebuild_git)
toolchain/CMakeLists.txt:31 (include)
-- Configuring incomplete, errors occurred!
So I'm not sure what we can do with this issue. Can you provide a reproducer for this crash that others can run?
from rust.
CharlesMengCA
commented on June 28, 2024
my bash
sudo pacman -S --noconfirm --needed \
git gyp mercurial ninja cmake ragel yasm nasm asciidoc enca \
gperf unzip p7zip gcc-multilib python-pip clang meson po4a \
python-mako python-j2cli python-jsonschema mold \
lld libc++ libc++abi less
cd
git clone https://github.com/shinchiro/mpv-winbuild-cmake.git --depth=1
cd mpv-winbuild-cmake
cmake -DTARGET_ARCH=x86_64-w64-mingw32 \
-DGCC_ARCH=x86-64-v3 \
-DSINGLE_SOURCE_LOCATION=$PWD/src_packages \
-DRUSTUP_LOCATION=$PWD/install_rustup \
-G Ninja -Bbuild64 -H.
cd build64
ninja rustup
from rust.
saethlin
commented on June 28, 2024
[root@b8419f69352e mpv-winbuild-cmake]# cmake -DTARGET_ARCH=x86_64-w64-mingw32 \
-DGCC_ARCH=x86-64-v3 \
-DSINGLE_SOURCE_LOCATION=$PWD/src_packages \
-DRUSTUP_LOCATION=$PWD/install_rustup \
-G Ninja -Bbuild64 -H.
-- The C compiler identification is GNU 14.1.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/sbin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- MINGW_INSTALL_PREFIX dir: /root/mpv-winbuild-cmake/build64/install/x86_64-w64-mingw32
-- SINGLE_SOURCE_LOCATION dir: /root/mpv-winbuild-cmake/src_packages
-- RUSTUP_LOCATION dir: /root/mpv-winbuild-cmake/install_rustup
-- Compiler used: gcc
-- Malloc: /usr/lib/libmimalloc.so
-- Ccache: OFF
-- Target toolchain to build: x86_64-w64-mingw32
CMake Error at cmake/custom_steps.cmake:130 (if):
if given arguments:
"EXISTS" "/root/mpv-winbuild-cmake/src_packages/mingw-w64" "GIT_CLONE_FLAGS" "--filter=tree" "0/.git"
Unknown arguments specified
Call Stack (most recent call first):
toolchain/mingw-w64.cmake:12 (force_rebuild_git)
toolchain/CMakeLists.txt:31 (include)
-- Configuring incomplete, errors occurred!
from rust.
saethlin
commented on June 28, 2024
Perhaps this is related? Something looks scuffed here
[root@b8419f69352e mpv-winbuild-cmake]# ls /root/mpv-winbuild-cmake/src_packages/
gcc-binutils gcc-wrapper 'mingw-w64;GIT_CLONE_FLAGS;--filter=tree;0'
from rust.
CharlesMengCA
commented on June 28, 2024
I don't have cmake problem (cmake version 3.29.5). here is my output
Same error in the rustup-install-err.log
-- The C compiler identification is GNU 14.1.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
patching file Modules/ExternalProject.cmake
Hunk #1 succeeded at 1288 (offset -9 lines).
Hunk #2 succeeded at 1328 (offset -9 lines).
Hunk #3 succeeded at 2844 (offset -3 lines).
Hunk #4 succeeded at 2888 (offset -3 lines).
Hunk #5 succeeded at 4067 (offset -3 lines).
patching file Modules/ExternalProject/gitclone.cmake.in
Hunk #1 succeeded at 61 (offset 12 lines).
-- MINGW_INSTALL_PREFIX dir: /home/cm/mpv-winbuild-cmake/build64/install/x86_64-w64-mingw32
-- SINGLE_SOURCE_LOCATION dir: /home/cm/mpv-winbuild-cmake/src_packages
-- RUSTUP_LOCATION dir: /home/cm/mpv-winbuild-cmake/install_rustup
-- Compiler used: gcc
-- Malloc: /usr/lib/libmimalloc.so
-- Ccache: OFF
-- Target toolchain to build: x86_64-w64-mingw32
-- Parsing 16 toolchain packages
-- Parsing 101 packages
-- Configuring done (6.3s)
-- Generating done (0.3s)
-- Build files have been written to: /home/cm/mpv-winbuild-cmake/build64
[7/9] Performing install step for 'rustup'
FAILED: toolchain/rustup-prefix/src/rustup-stamp/rustup-install /home/cm/mpv-winbuild-cmake/build64/toolchain/rustup-prefix/src/rustup-stamp/rustup-install
cd /home/cm/mpv-winbuild-cmake/build64/toolchain/rustup-prefix/src/rustup-build && /usr/bin/cmake -P /home/cm/mpv-winbuild-cmake/build64/toolchain/rustup-prefix/src/rustup-stamp/rustup-install-.cmake && /usr/bin/cmake -E touch /home/cm/mpv-winbuild-cmake/build64/toolchain/rustup-prefix/src/rustup-stamp/rustup-install
CMake Error at /home/cm/mpv-winbuild-cmake/build64/toolchain/rustup-prefix/src/rustup-stamp/rustup-install-.cmake:49 (message):
Command failed: 101
'/home/cm/mpv-winbuild-cmake/build64/exec' 'cargo' 'install' 'cargo-c' '--profile=release-strip' '--features=vendored-openssl'
See also
/home/cm/mpv-winbuild-cmake/build64/toolchain/rustup-prefix/src/rustup-stamp/rustup-install-*.log
ninja: build stopped: subcommand failed.
from rust.
CharlesMengCA
commented on June 28, 2024
Thank you very much! when I removed the export LD_PRELOAD="@MALLOC@" from exec.in. The problem was gone.
Arch Linux upgraded mimalloc to 2.1.7-1 on June 8. I will try to downgrade it and let you know if the problem is also gone.
from rust.
CharlesMengCA
commented on June 28, 2024
Downgraded to mimalloc-2.1.2 (https://archive.archlinux.org/repos/2024/06/07/extra/os/x86_64/mimalloc-2.1.2-1-x86_64.pkg.tar.zst), kept export LD_PRELOAD="@MALLOC@" and the build/install succeeded.
Rust is not happy with mimalloc 2.1.7 in Arch Linux. Where should I report the bug, here, Arch Linux or mimalloc?
from rust.
saethlin
commented on June 28, 2024
gdb has a much more informative backtrace (as usual) than our segfault handler:
#0 atomic_load_p (a=0x140080, mo=atomic_memory_order_relaxed) at include/jemalloc/internal/atomic.h:83
#1 rtree_leaf_elm_bits_read (elm=0x140080, dependent=true, tsdn=<optimized out>, rtree=<optimized out>) at include/jemalloc/internal/rtree.h:182
#2 rtree_leaf_elm_read (elm=0x140080, dependent=true, tsdn=<optimized out>, rtree=<optimized out>) at include/jemalloc/internal/rtree.h:242
#3 rtree_metadata_read (tsdn=tsdn@entry=0x70709f1ffc78, rtree=<optimized out>, rtree_ctx=rtree_ctx@entry=0x70709f1ffe28, key=key@entry=2879372984448) at include/jemalloc/internal/rtree.h:446
#4 0x000057a360aca407 in emap_alloc_ctx_lookup (tsdn=0x70709f1ffc78, ptr=0x29e68010080, emap=<optimized out>, alloc_ctx=<optimized out>) at include/jemalloc/internal/emap.h:238
#5 ifree (tsd=0x70709f1ffc78, ptr=0x29e68010080, tcache=0x70709f1fffd0, slow_path=false) at src/jemalloc.c:2877
#6 _rjem_je_free_default (ptr=0x29e68010080) at src/jemalloc.c:3014
#7 0x00007070a6838c8e in llvm::sys::DontRemoveFileOnSignal(llvm::StringRef) () from /home/ben/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.81.0-nightly
#8 0x00007070a68393cc in llvm::sys::fs::TempFile::keep(llvm::Twine const&) () from /home/ben/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.81.0-nightly
#9 0x00007070a683931b in llvm::writeArchive(llvm::StringRef, llvm::ArrayRef<llvm::NewArchiveMember>, llvm::SymtabWritingMode, llvm::object::Archive::Kind, bool, bool, std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer> >, bool) ()
from /home/ben/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.81.0-nightly
#10 0x00007070ad824045 in LLVMRustWriteArchive.warm () from /home/ben/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-9b2f1304048a4118.so
#11 0x00007070ad307761 in <rustc_codegen_llvm::back::archive::LlvmArchiveBuilder as rustc_codegen_ssa::back::archive::ArchiveBuilder>::build () from /home/ben/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-9b2f1304048a4118.so
#12 0x00007070ad496f32 in rustc_codegen_ssa::back::link::link_binary () from /home/ben/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-9b2f1304048a4118.so
#13 0x00007070ad4967a2 in <rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_ssa::traits::backend::CodegenBackend>::link () from /home/ben/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-9b2f1304048a4118.so
#14 0x00007070ad49a064 in <rustc_interface::queries::Linker>::link () from /home/ben/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-9b2f1304048a4118.so
#15 0x00007070ad215c6d in rustc_interface::interface::run_compiler::<core::result::Result<(), rustc_span::ErrorGuaranteed>, rustc_driver_impl::run_compiler::{closure#0}>::{closure#1} ()
from /home/ben/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-9b2f1304048a4118.so
#16 0x00007070ad1d2a49 in std::sys_common::backtrace::__rust_begin_short_backtrace::<rustc_interface::util::run_in_thread_with_globals<rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_span::ErrorGuaranteed>, rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#0}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#0}::{closure#0}, core::result::Result<(), rustc_span::ErrorGuaranteed>> () from /home/ben/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-9b2f1304048a4118.so
#17 0x00007070ad1d2800 in <<std::thread::Builder>::spawn_unchecked_<rustc_interface::util::run_in_thread_with_globals<rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_span::ErrorGuaranteed>, rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#0}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#0}::{closure#0}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#2} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} () from /home/ben/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-9b2f1304048a4118.so
#18 0x00007070ae930e4b in alloc::boxed::{impl#48}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:2063
#19 alloc::boxed::{impl#48}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:2063
#20 std::sys::pal::unix::thread::{impl#2}::new::thread_start () at library/std/src/sys/pal/unix/thread.rs:108
#21 0x00007070a7ea6ded in start_thread (arg=<optimized out>) at pthread_create.c:447
#22 0x00007070a7f2a0dc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
Our libLLVM.so is trying to deallocate memory with jemalloc, which was almost certainly allocated with mimalloc. I strongly suspect that on any system attempting to use LD_PRELOAD to override the allocator with anything will cause a similar explosion.
from rust.
saethlin
commented on June 28, 2024
rustc version seems entirely irrelevant; I can reproduce this on compilers at least back to 1.50 which is from 2021. I am surprised that the mimalloc version matters. This looks like plain old allocator confusion, but if that were the case the mimalloc version shouldn't matter, and it shouldn't really matter which alternative allocator is used.
from rust.
mati865
commented on June 28, 2024
I think the only way jemalloc could deallocate this memory is when deallocation is performed by rustc (jemalloc is statically linked in rustc binary).
It's hard to imagine how that could happen; rustc would have to allocate memory (using jemalloc), LLVM would have to move and mutate it (basically reallocate with mimalloc), and finally, rustc would deallocate it with jemalloc.
The same path would be performed with system allocator, but maybe jemalloc can handle such a case?
If that is anywhere close to the truth, the fact mimalloc 2.1.2 (and possibly earlier) is a pure coincidence and such usage is basically impossible to support.
Maybe @Kobzol who had an idea to use jemalloc with LLVM library (see #122047) has any idea?
from rust.
Kobzol
commented on June 28, 2024
Possibly some allocations go to jemalloc directly, while some go through malloc/free and are overridden with LD_PRELOAD. rustc uses jemalloc in a weird way, and probably any attempts to override the allocator with LD_PRELOAD` will be problematic.
from rust.
Andarwinux
commented on June 28, 2024
I'm surprised that rustc's llvm is not compiled with jemalloc. If I understand correctly, if there exists some system whose libc malloc implementation happens to be mimalloc, then this crash will also occur?
from rust.
Related Issues (20)
- Repeated spurious rustdoc test failure: rustdoc-gui/help-page HOT 1
- Panic when setting RUST_MIN_STACK incorrectly HOT 3
- Docs produce invalid type when using bounds in associated type position HOT 1
- `unnameable_types` should show where the type is reachable HOT 2
- MIR_borrow_checking and MIR_effect_checking are very slow HOT 2
- Bevy segfaults on some Windows systems after Rust 1.79 update HOT 24
- ICE with generic_const_exprs (and std::simd)
- recommending the wrong Arbitrary
- Complete tier 3 target information HOT 11
- Inappropriate `unused_parens` for inline `const` inside `vec!`/macros HOT 1
- Inline `const` expression inside `vec!`/macros should suggest adding parentheses under edition <2024 HOT 2
- ICE: layout: :evergreen_tree: : `Size != Size` HOT 5
- Running run-make tests performs unnecessary rebuilds HOT 7
- this function depends on never type fallback being `()` HOT 9
- please update src/stage0 for beta branch HOT 3
- An unused output of some macros in std (e.g., `format!()`) should produce a unused_must_use warning.
- Tracking Issue for async drop codegen HOT 1
- Error tainting in parallel compiler HOT 2
- Cargo check internal error (using UOM library) HOT 3
- document for E0502 insufficiently general
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rust.