intelxed / mbuild Goto Github PK

In this file https://github.com/intelxed/mbuild/blob/master/mbuild/env.py#L445 a function platform.linux_distribution is used. According to the python documentation at https://docs.python.org/3.7/library/platform.html#platform.linux_distribution:

Deprecated since version 3.5, will be removed in version 3.8: See alternative like the distro package.

And indeed on my system with Python 3.8 there's no such function.

I am currently working on a project where we are using xed to decode x86 binaries.
Now we are trying to expand our support to RISC-V (as host architecture).

But we noticed that xed is not able to be built on RISC-V, because normalize_cpu_name does not understand riscv64.

Simply adding another branch for "riscv64" here fixes it according to my testing:

On Microsoft Windows with VC 11.0, msvs.py assumes cl.exe is in the following directory:

C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\bin\amd64

However, things changed on VC 11.0 and now compiler binaries for AMD64 are under:

C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\bin\x86_amd64

Notice the change in the last component of the directory.

I can commit a patch, but I would like to study the source first so that my solution is elegant and compatible with how mbuild works. If you have a faster solution for this let me know :)

This relates to #258 of xed
I can always reproduce the issue using the latest VS2022/MSVC
The toolchain is resolved incorrectly (by build_env.py/msvs.py) to C:/Program Files/Microsoft Visual Studio/2022/Community/VC/bin/amd64/cl.exe instead of C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.31.31103/bin/Hostx64/x64/

Using: MSVC 19.31.31104.0

Race condition!?

Install anaconda:

(asv) rscohn1@fxsatlin01$ wget -q https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh
(asv) rscohn1@fxsatlin01$ chmod +x Miniconda2-latest-Linux-x86_64.sh
(asv) rscohn1@fxsatlin01$ ./Miniconda2-latest-Linux-x86_64.sh -b -p ~/tmp/mymini

Download mbuild and install it:

(asv) rscohn1@fxsatlin01$ git clone https://github.com/intelxed/mbuild.git
Cloning into 'mbuild'...
remote: Counting objects: 66, done.
remote: Compressing objects: 100% (44/44), done.
remote: Total 66 (delta 18), reused 65 (delta 17), pack-reused 0
Unpacking objects: 100% (66/66), done.
Checking connectivity... done.
(asv) rscohn1@fxsatlin01$ /tmp/mymini/bin/pip install -e mbuild
Obtaining file:///localdisk/work/rscohn1/workspaces/mbuild
Installing collected packages: mbuild
Running setup.py develop for mbuild
Successfully installed mbuild-0.2496
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.`

3.py test randomly fails. Looks like you are not waiting for all threads to end?

(asv) rscohn1@fxsatlin01$ /tmp/mymini/bin/python 3.py
[INVOKED] 3.py
[START TIME] 2016-12-19 09_36_33 CST
[CURRENT DIRECTORY] /localdisk/work/rscohn1/workspaces/mbuild/tests
[UNAME] ('Linux', 'fxsatlin01', '3.13.0-85-generic', '#129-Ubuntu SMP Thu Mar 17 20_50_15 UTC 2016', 'x86_64', 'x86_64')
[SYSTEM] Linux
[HOSTNAME] fxsatlin01
[BUILD_OS] lin
[BUILD_CPU] x86-64
[HOST_OS] lin
[HOST_CPU] x86-64
[HOST_CPU] x86-64
[BUILD] succeeded
(asv) rscohn1@fxsatlin01$ /tmp/mymini/bin/python 3.py
[INVOKED] 3.py
[START TIME] 2016-12-19 09_36_35 CST
[CURRENT DIRECTORY] /localdisk/work/rscohn1/workspaces/mbuild/tests
[UNAME] ('Linux', 'fxsatlin01', '3.13.0-85-generic', '#129-Ubuntu SMP Thu Mar 17 20_50_15 UTC 2016', 'x86_64', 'x86_64')
[SYSTEM] Linux
[HOSTNAME] fxsatlin01
[BUILD_OS] lin
[BUILD_CPU] x86-64
[HOST_OS] lin
[HOST_CPU] x86-64
[HOST_CPU] x86-64
[BUILD] succeeded
(asv) rscohn1@fxsatlin01$ /tmp/mymini/bin/python 3.py
[INVOKED] 3.py
[START TIME] 2016-12-19 09_36_36 CST
[CURRENT DIRECTORY] /localdisk/work/rscohn1/workspaces/mbuild/tests
[UNAME] ('Linux', 'fxsatlin01', '3.13.0-85-generic', '#129-Ubuntu SMP Thu Mar 17 20_50_15 UTC 2016', 'x86_64', 'x86_64')
[SYSTEM] Linux
[HOSTNAME] fxsatlin01
[BUILD_OS] lin
[BUILD_CPU] x86-64
[HOST_OS] lin
[HOST_CPU] x86-64
[HOST_CPU] x86-64
[BUILD] succeeded
Exception in thread Thread-4 (most likely raised during interpreter shutdown):
(asv) rscohn1@fxsatlin01$

Work in progress

rscohn2@d303ecb

still some issues with unicode

See https://bugs.python.org/issue42189.

I guess you could either work around this or wait to see if Python 3.9 is fixed. In the meantime Xed won't build on platforms that are shipping Python 3.9 (e.g. Fedora 33).

Hi there,

I just tryed to build xed on a Windows 10 machine with Visual Studio 2022 (v17.4.1):

[MBUILD WARNING] Command execution failed. Waiting for remaining jobs and exiting.
R: 0 P: 110 C: 4 E: 4 / 129 msecs
[COMMAND     ] "C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/bin/amd64/cl.exe" -I../include/private -I../include/public/xed -I../include/public -Iobj -Iobj/include-private     /nologo  /MT  /favor:EM64T  /W4 /WX /wd4091 /wd4127 /wd4505 /wd4702 /wd4244 /wd4292 /DXED_VERSION="v2022.10.11" /DXED_AMD_ENABLED /DXED_VIA_ENABLED /DXED_AVX /DXED_SUPPORTS_AVX512 /DXED_MPX /DXED_CET /DXED_SUPPORTS_SHA /DXED_SUPPORTS_WBNOINVD /DXED_DECODER /DXED_ENCODER /DXED_SUPPORTS_LZCNT_TZCNT /DXED_BUILD /c /Foobj/xed-reg-enum.obj D:/Development/intelxed/xed/Build64/obj/xed-reg-enum.c
[EXIT_STATUS ] 399
[STDERR]

COMMAND ENCOUNTERD AN EXCEPTION
Traceback (most recent call last):
  File "D:\Development\intelxed\xed\Build64\..\..\mbuild\mbuild\util.py", line 1043, in run
    self.sub = subprocess.Popen(cmd_args,
  File "C:\Program Files\Python310\lib\subprocess.py", line 966, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Program Files\Python310\lib\subprocess.py", line 1435, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2]

It seems like the directory structure has changed and C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/bin/amd64/cl.exe does not exist anymore in this version.

I was able to find this executable here instead:

C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.34.31933\bin\Hostx64\x64