0.29.0: install target uses hardcoded install directory for python modules about bcc HOT 15 CLOSED

I would recommend making another release since build issues that prevent people from successfully installing the software tend to be a pretty thorny problem for affected users.

Two months is a bit on the long side, for that.

from bcc.

Sorry. the tag is incorrect, it should be v0.29.0. could you sync with latest bcc repo and try again?

from bcc.

I have exactly the same problem on Gentoo using v0.29.0; python stuff is installed into /lib/python3.x/ (where it does not belong) and is not found by any tools.

from bcc.

Could you help bisect which commit caused this issue?

from bcc.

The problem is caused by #6813fbce ("Allow for installing python as a non-system package").
Building 0.29.0-final with only that commit reverted installs python stuff where it should go, just like with 0.28.

from bcc.

Could you help bisect which commit caused this issue?

OK. One sec ..

from bcc.

@hhoffstaette @kloczek Could you help investigate how we could fix the issue at the same time preserving the functionality as in 6813fbc If needed we can add a cmake option for that. thanks!

from bcc.

Issue is that 0.29.0 tag is without dist tar ball asset so I cannot plug that tag into my existing build procedure which I can request to execute on my build infra.

from bcc.

@kloczek https://github.com/iovisor/bcc/releases/tag/v0.29.0 there is a tarball with submodules included.

from bcc.

@hhoffstaette @kloczek Could you help investigate how we could fix the issue at the same time preserving the functionality as in 6813fbc If needed we can add a cmake option for that. thanks!

I think that ideally you would stop calling one build system from another build system, but instead use one build system that knows how to handle both C/C++ and python.

Using setuptools isn't even a long-term solution either way, because ${PY_CMD} setup.py install ... is going to stop working when setuptools finally does what it's been warning it will do for a while now, and removes the ability to execute setup.py as a script. I'm not aware of a specific timeframe for this but it is something that they want to do at some point.

EDIT: for context, you can install a whole bunch more packages as well and then use those to install a setuptools-based wheel. However, this is a great deal of effort and additional dependencies for, ultimately, at the end of the day -- nothing other than "install the bcc/ directory to the correct site-packages".

Even the setuptools-produced .egg-info / .dist-info metadata is effectively useless, since it's not the same as PyPI.

from bcc.

@hhoffstaette @kloczek Could you help investigate how we could fix the issue at the same time preserving the functionality as in 6813fbc If needed we can add a cmake option for that. thanks!

The puzzling thing is that the existing construct if(NOT PYTHON_PREFIX) should work, but somehow does not. So I took a closer look at the conditional assignment and saw the comma between the variables, which looked fishy to me. I removed it (see below) and now it seems to work as expected. 🎉
Can someone more versed in CMake confirm that this won't break anything else?
Maybe @leeavital can chime in. If this is indeed correct I'll make a PR.

diff --git a/src/python/CMakeLists.txt b/src/python/CMakeLists.txt
index a897aa98..7f5b103e 100644
--- a/src/python/CMakeLists.txt
+++ b/src/python/CMakeLists.txt
@@ -38,7 +38,7 @@ foreach(PY_CMD ${PYTHON_CMD})
   add_custom_target(bcc_py_${PY_CMD_ESCAPED} ALL DEPENDS ${PIP_INSTALLABLE})
 
   if(NOT PYTHON_PREFIX)
-     set(PYTHON_PREFIX, ${CMAKE_INSTALL_PREFIX} )
+     set(PYTHON_PREFIX ${CMAKE_INSTALL_PREFIX})
   endif()
 
   install(

from bcc.

Can someone more versed in CMake confirm that this won't break anything else?

Using commas is incorrect CMake syntax. The Meson build system would require it, on the other hand. :P

Interestingly, you can see what cmake is actually doing here if you set it as a CACHE variable. The variable name is PYTHON_DIR,, but you cannot actually access such a variable after setting it, at least... not with ${} lookup syntax. It still works via some other methods not used here.

Unfortunately, cmake as a stringly-typed programming language has no innate protection against this kind of issue.

from bcc.

@hhoffstaette @eli-schwartz @kloczek @rtomayko Thanks for all of you trying to fix the problem! Just merged to the master branch. Do you think we need another release (e.g., v0.29.1) containing the fix? Or we can wait for another say two months for the next release (e.g., v0.30.0)?

from bcc.

Sounds good. Will cut another minor release tomorrow.

from bcc.

Sounds good. Will cut another minor release tomorrow.

Appreciate it, but please also add the python-3.12 fix in #4832 - thanks :)

from bcc.