Hash table capacity overflow on x86_64 FreeBSD about maturin HOT 10 CLOSED

Can you reproduce it by running a debug version of maturin so we can get a backtrace with line information? Clone the repository and run

cargo run build -i /usr/local/bin/python3.9 -m <path to rpds-py Cargo.toml>

from maturin.

FYI I'm unable to reproduce it on aarch64 FreeBSD 13.2, and I don't have a x86_64 machine so I can't investigate further.

from maturin.

I have no experience with cargo or maturin. A simple git clone of the maturin repo and "cargo run build ...." is enough?

Would it be ok to build a debug version of maturin out of the 1.2.0 release? If yes, how would I do that (something to add to the cargo build args?)? That would be easy for me, I could modify the FreeBSD port metadata (either by adding env variables for the build, or args for cargo, or patching files before the build) to generate the debug build inside the package building infrastructure and use it in the same way which generates the issue. If I simply install the prerequisites of maturin and rdps-py I will not have the same operation environment as the package building does (it runs in a very pristine environment, newly created for each build, every normal system is "polluted" compared to that).

from maturin.

A simple git clone of the maturin repo and "cargo run build ...." is enough?

Yes, provided you have rust installed.

Would it be ok to build a debug version of maturin out of the 1.2.0 release?

Sure, probably just need to add debug=True to

RustBin("maturin", args=cargo_args, cargo_manifest_args=["--locked"], debug=True)

from maturin.

py39-rpds-py-0.9.2.log

This is the complete build log with the debug version of maturin. The backtrace is at the end.

from maturin.

Thanks, the backtrace points the panic to deserialization of pyproject.toml with toml_edit, which might indicate bug in toml_edit so it's not directly fixable in maturin.

Can you also try the main branch using cargo run build -i /usr/local/bin/python3.9 -m <path to rpds-py Cargo.toml> to see if it also occurs on main?

from maturin.

How do you create an official release from the git repo? I would like to generate a file like an official release to be able to use it inside the package building infrastructure. This way I can be sure I have the same execution environment.

from maturin.

You can run maturin sdist (using maturin 1.2.x) on main branch to generate a source distribution for maturin itself.

from maturin.

Yes, the main branch as of today has the same problem. At the same time we noticed a bug in some string functions in FreeBSD (they got a SIMD optimisation recently). A fix affecting very long buffer sizes for memchr() to FreeBSD was committed yesterday. As toml is a string-heavy thing, it looks to me the likelyhood of this bug affecting toml is >0... at least it is worth a look to check if it is causing this.

I think we can close this issue, as it is in a depdendency of maturin.

from maturin.

I found the issue which caused this.
I do a cross-build of maturin for another system. The build system is an Intel Xeon. The target system is a slow AMD Athlon system. I compile for the target with cputype amdfam10 and with -fvectorize. maturin (with target amdfam10 and -fvectorize) is then run in the build on the Intel build system. If I exclude the cputype and the vectorizing switch for the maturin build, the use of it on the build system works. I have not tested if only not using one of those is enough.

So this is a compiler options + CPU arch issue, not really a code issue.

from maturin.