Conflict with Houdini USD and MoonRay USD? about openmoonray HOT 15 OPEN

IT WORKS!!!

from openmoonray.

Made some progress by symlinking all the "libusd" libraries to the "libpxr" ones that come with Houdini. That eliminated 99% of the errors. Still not working though. Now we're left with:

Error in 'pxrInternal_v0_22pxrReserved::PlugPlugin::_Load' at line 257 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/base/plug/plugin.cpp : 'Failed to load plugin 'hdMoonray': /installs/openmoonray/plugin/hdMoonray.so: undefined symbol: _ZN32pxrInternal_v0_22pxrReserved14TfPyObjWrapperC1EN5boost6python3api6objectE in '/installs/openmoonray/plugin/hdMoonray.so''
Error in 'pxrInternal_v0_22pxrReserved::HdRendererPluginRegistry::CreateRenderDelegate' at line 100 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rendererPluginRegistry.cpp : 'Couldn't find plugin for id HdMoonrayRendererPlugin'

Honestly this is all kind of bananas. Is there anybody at DreamWorks who could lend a helping hand? I don't think this should be this complicated and I feel like we're all flying blind out here. Thanks!

from openmoonray.

@BrianHanke
the

hdMoonray.so: undefined symbol: _ZN32pxrInternal_v0_22pxrReserved14TfPyObjWrapperC1EN5boost6python3api6objectE in '/installs/openmoonray/plugin/hdMoonray.so

makes me think of an API mismatch between libraries.

My first suggestion would be to look at which version of USD Houdini is using. For example, 19.5 is on 22.05: https://www.sidefx.com/docs/houdini/licenses/index.html .
Then, check if moonray is using the same version.
According to

In that case, I think that it would be good to have a dedicated guide on how to build Moonray as a DCC plugin, since in that scenario it would be safer/easier to rely on the USD and OCIO offered by the DCC, instead of having the build script compile and link to a different USD. As long as openmoonray and the DCC are on the same VFX reference platform year, most of the versions of these big libraries should be in line (even though sadly that beast of USD isn't yet on the VFX ref platform, afaik..)

from openmoonray.

API mismatch between libraries.

Yes indeed, that's my conclusion too. I tried one more test, symlinking all the Houdini libraries before the final MR build step. No library errors when doing that, but I started getting syntax/API problems.

SideFX recommended that I build against the Houdini USD libraries and the professional coders I've talked to agree. It's obviously a big task and above my skill level. Not much to do until DreamWorks throws us a bone...

from openmoonray.

There are a couple of possible solutions. You should be able to get USD from the sidefx repo (https://github.com/sideeffects/USD) instead of pxr by changing the location in the dependency CMake project. You'll have to change the git tag, but you can use symbolic tags like 'v22.05a' -- there also seem to be tags for some specific Houdini versions on that repo.

Another way, that is more involved, is to use the USD libs and headers shipped with Houdini. To do this, you need a pxrConfig.cmake that targets these. I've uploaded a tar file with a modified set of config files and a README to: https://docs.openmoonray.org/assets/build_extras/moonray_houdini_pxr_config.tar -- I think you should be able to access this. You'll have to modify the CMake preset used to build MoonRay itself to get this working.

from openmoonray.

No, dwa-environment is only used by internal dwa builds, so you don't need to change it.

from openmoonray.

@nospitters Definitely. I'm working on a guide right now, will probably be ready this weekend. Check out my GitHub to stay updated.

from openmoonray.

@rgwilson68 Excellent, thanks! Will check this out over the next few days.

from openmoonray.

Made a new build with the changes in option number 2 above. Build went fine, but still no rendering. Houdini crashes after clicking on MoonRay with the following log. usdview throws even more errors, a sample of which I put further down. It's just the same few ones over and over, but hundreds if not thousands of lines.

Crash report from brian; Houdini FX Version 19.5.534 [linux-x86_64-gcc9.3]
Uptime 18 seconds
Mon Mar 27 17:42:54 2023
Caught signal 11

Traceback from 29463 ThreadId=0x7f683ca39700
AP_Interface::si_CrashHandler::handledSignal(UTsignalHandlerArg) <libHoudiniUI.so>
signalCallback(UTsignalHandlerArg) <libHoudiniUT.so>
UT_Signal::UT_ComboSignalHandler::operator()(int, siginfo_t*, void*) const <libHoudiniUT.so>
UT_Signal::processSignal(int, siginfo_t*, void*) <libHoudiniUT.so>
L_unlock_13 <libpthread.so.0>
hdMoonray::Light::Sync(pxrInternal_v0_22__pxrReserved_::HdSceneDelegate*, pxrInternal_v0_22__pxrReserved__::HdRenderParam*, unsigned int*) <hdMoonray.so>
pxrInternal_v0_22__pxrReserved__::Hd_PrimTypeIndex<pxrInternal_v0_22__pxrReserved__::HdBprim>::RenderDelegateDestroyPrim(pxrInternal_v0_22__pxrReserved_::HdRenderDelegate*, pxrInternal_v0_22__pxrReserved__::HdBprim*) <libpxr_hd.so>
pxrInternal_v0_22__pxrReserved__::HdRenderIndex::SyncAll(std::vector<std::shared_ptr<pxrInternal_v0_22__pxrReserved__::HdTask>, std::allocator<std::shared_ptr<pxrInternal_v0_22__pxrReserved__::HdTask> > >, std::unordered_map<pxrInternal_v0_22__pxrReserved__::TfToken, pxrInternal_v0_22__pxrReserved__::VtValue, pxrInternal_v0_22__pxrReserved__::TfToken::HashFunctor, std::equal_to<pxrInternal_v0_22__pxrReserved__::TfToken>, std::allocator<std::pair<pxrInternal_v0_22__pxrReserved__::TfToken const, pxrInternal_v0_22__pxrReserved__::VtValue> > >) <libpxr_hd.so>
pxrInternal_v0_22__pxrReserved__::XUSD_ImagingEngineGL::DispatchRender(UT_StringHolder const&, pxrInternal_v0_22__pxrReserved__::UsdPrim const&, pxrInternal_v0_22__pxrReserved__::XUSD_ImagingRenderParams const&) <USD_UI.so>
HUSD_Imaging::updateRenderData(UT_Matrix4T const&, UT_Matrix4T const&, UT_Rect<UT_DimRectImpl> const&, bool) <libHoudiniUSD.so>
tbb::internal::function_task<HUSD_Imaging::launchBackgroundRender(UT_Matrix4T const&, UT_Matrix4T const&, UT_Rect<UT_DimRectImpl> const&, UT_StringRef const&, UT_Options const*, bool)::{lambda()#1}>::execute() <libHoudiniUSD.so>
tbb::internal::custom_schedulertbb::internal::IntelSchedulerTraits::process_bypass_loop(tbb::internal::context_guard_helper&, tbb::task*, long) (custom_scheduler.h:474)
tbb::internal::custom_schedulertbb::internal::IntelSchedulerTraits::local_wait_for_all(tbb::task&, tbb::task*) (custom_scheduler.h:636)
tbb::internal::arena::process(tbb::internal::generic_scheduler&) (arena.cpp:196)
tbb::internal::market::process(rml::job&) (market.cpp:667)
tbb::internal::rml::private_worker::run() (private_server.cpp:266)
tbb::internal::rml::private_worker::thread_routine(void*) (private_server.cpp:219)
start_thread <libpthread.so.0>
__clone <libc.so.6>

Traceback from 29463 ThreadId=0x7f68674fff40
AP_Interface::coreDumpChaser(UTsignalHandlerArg) <libHoudiniUI.so>
AP_Interface::si_CrashHandler::chaser(UTsignalHandlerArg) <libHoudiniUI.so>
signalCallback(UTsignalHandlerArg) <libHoudiniUT.so>
UT_Signal::UT_ComboSignalHandler::operator()(int, siginfo_t*, void*) const <libHoudiniUT.so>
UT_Signal::processSignal(int, siginfo_t*, void*) <libHoudiniUT.so>
_L_unlock_13 <libpthread.so.0>
_GI_ppoll <libc.so.6>
qt_safe_poll(pollfd*, unsigned long, timespec const*) <libQt5Core.so.5>
QEventDispatcherUNIX::processEvents(QFlagsQEventLoop::ProcessEventsFlag) <libQt5Core.so.5>
QXcbNativeInterface::dumpNativeWindows(unsigned long long) const <libQt5XcbQpa.so.5>
UI_QtGeneratorThread::processEvents(QFlagsQEventLoop::ProcessEventsFlag) <libHoudiniUI.so>
UI_QtGeneratorThread::waitForRawEvents() <libHoudiniUI.so>
UI_Queue::waitForGenerators(int) <libHoudiniUI.so>
UI_Queue::waitForEvents() <libHoudiniUI.so>
UI_Queue::generateEvents(bool, bool) <libHoudiniUI.so>
UI_Queue::eventLoop() <libHoudiniUI.so>
main <libHoudiniUI.so>
__libc_start_main <libc.so.6>
main

Error in 'pxrInternal_v0_22__pxrReserved__::TfType::DefineCppType' at line 963 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/base/tf/type.cpp : 'TfType 'SdfVariantSpec' already has a defined C++ type; cannot redefine'
Error in 'pxrInternal_v0_22__pxrReserved_::Vt_CastRegistry::Register' at line 137 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/base/vt/value.cpp : 'VtValue cast already registered from 'string' to 'SdfAssetPath'. New cast will be ignored.'
Error in 'pxrInternal_v0_22__pxrReserved__::Vt_CastRegistry::Register' at line 137 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/base/vt/value.cpp : 'VtValue cast already registered from 'double' to 'SdfTimeCode'. New cast will be ignored.'
Error in 'pxrInternal_v0_22__pxrReserved__::Vt_CastRegistry::Register' at line 137 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/base/vt/value.cpp : 'VtValue cast already registered from 'SdfTimeCode' to 'double'. New cast will be ignored.'
USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found'
Error in 'pxrInternal_v0_22__pxrReserved__::HdRprim::ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22__pxrReserved_::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found'
Error in 'pxrInternal_v0_22__pxrReserved__::HdRprim::ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22__pxrReserved_::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found'
Error in 'pxrInternal_v0_22__pxrReserved__::HdRprim::ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22__pxrReserved_::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found'
Error in 'pxrInternal_v0_22__pxrReserved__::HdRprim::ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22__pxrReserved_::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found'
Error in 'pxrInternal_v0_22__pxrReserved__::HdRprim::ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22__pxrReserved_::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found'
Error in 'pxrInternal_v0_22__pxrReserved__::HdRprim::ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22__pxrReserved_::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found'
Error in 'pxrInternal_v0_22__pxrReserved__::HdRprim::ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22__pxrReserved_::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found'
Error in 'pxrInternal_v0_22__pxrReserved__::HdRprim::ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22__pxrReserved_::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found'
Error in 'pxrInternal_v0_22__pxrReserved__::HdRprim::ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22__pxrReserved_::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found'
Error in 'pxrInternal_v0_22__pxrReserved__::HdRprim::ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22__pxrReserved_::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found'
Error in 'pxrInternal_v0_22__pxrReserved__::HdRprim::ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22__pxrReserved_::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found'
Error in 'pxrInternal_v0_22__pxrReserved__::HdRprim::ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22__pxrReserved_::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found'
Error in 'pxrInternal_v0_22__pxrReserved__::HdRprim::ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22__pxrReserved_::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found'
Error in 'pxrInternal_v0_22__pxrReserved__::HdRprim::ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22__pxrReserved_::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found'
Error in 'pxrInternal_v0_22__pxrReserved__::HdRprim::ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22__pxrReserved_::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found'
Error in 'pxrInternal_v0_22__pxrReserved__::Tf_PyOwnershipRefBaseUniqueChanged' at line 402 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/base/tf/pyIdentity.cpp : 'Couldn't get uniqueId associated with refBase!'

from openmoonray.

Is it possible that the pxr USD build is still on the system, and being picked up by cmake? You might need to check under /installs and /usr/local for pxrConfig.cmake, and remove it...

from openmoonray.

Ah ok, I deleted all that and all the libusd* files, but I didn't see pxrConfig.cmake. Trying again!

from openmoonray.

Do I need to delete PXR_USD_LOCATION and PXR_INCLUDE_DIRS under both container-environment and dwa-environment?

from openmoonray.

Thanks so much for your amazing help Rob, much appreciated!

from openmoonray.

@BrianHanke congratulations...
would it be possible to share how you got it work?
maybe build instructions?

from openmoonray.

Crash report from yu; Houdini FX Version 19.5.534 [linux-x86_64-gcc9.3]
Uptime 135 seconds
Sun Apr 9 16:55:40 2023
Caught signal 11

Traceback from 6705 ThreadId=0x7f4e87fff700
AP_Interface::si_CrashHandler::handledSignal(UTsignalHandlerArg) <libHoudiniUI.so>
signalCallback(UTsignalHandlerArg) <libHoudiniUT.so>
UT_Signal::UT_ComboSignalHandler::operator()(int, siginfo_t*, void*) const <libHoudiniUT.so>
UT_Signal::processSignal(int, siginfo_t*, void*) <libHoudiniUT.so>
L_unlock_13 <libpthread.so.0>
pxrInternal_v0_22__pxrReserved_::HdChangeTracker::MarkSprimDirty(pxrInternal_v0_22__pxrReserved__::SdfPath const&, unsigned int) <libpxr_hd.so>
hdMoonray::RenderDelegate::setDisableLighting(bool) <hdMoonray.so>
hdMoonray::RenderSettings::apply() <hdMoonray.so>
hdMoonray::RenderDelegate::getRendererApplySettings() <hdMoonray.so>
hdMoonray::RenderBuffer::Allocate(pxrInternal_v0_22__pxrReserved__::GfVec3i const&, pxrInternal_v0_22__pxrReserved__::HdFormat, bool) <hdMoonray.so>
pxrInternal_v0_22__pxrReserved__::HdRenderBuffer::Sync(pxrInternal_v0_22__pxrReserved__::HdSceneDelegate*, pxrInternal_v0_22__pxrReserved__::HdRenderParam*, unsigned int*) <libpxr_hd.so>
pxrInternal_v0_22__pxrReserved__::Hd_PrimTypeIndex<pxrInternal_v0_22__pxrReserved__::HdBprim>::RenderDelegateDestroyPrim(pxrInternal_v0_22__pxrReserved_::HdRenderDelegate*, pxrInternal_v0_22__pxrReserved__::HdBprim*) <libpxr_hd.so>
pxrInternal_v0_22__pxrReserved__::HdRenderIndex::SyncAll(std::vector<std::shared_ptr<pxrInternal_v0_22__pxrReserved__::HdTask>, std::allocator<std::shared_ptr<pxrInternal_v0_22__pxrReserved__::HdTask> > >, std::unordered_map<pxrInternal_v0_22__pxrReserved__::TfToken, pxrInternal_v0_22__pxrReserved__::VtValue, pxrInternal_v0_22__pxrReserved__::TfToken::HashFunctor, std::equal_to<pxrInternal_v0_22__pxrReserved__::TfToken>, std::allocator<std::pair<pxrInternal_v0_22__pxrReserved__::TfToken const, pxrInternal_v0_22__pxrReserved__::VtValue> > >) <libpxr_hd.so>
pxrInternal_v0_22__pxrReserved__::XUSD_ImagingEngineGL::DispatchRender(UT_StringHolder const&, pxrInternal_v0_22__pxrReserved__::UsdPrim const&, pxrInternal_v0_22__pxrReserved__::XUSD_ImagingRenderParams const&) <USD_UI.so>
HUSD_Imaging::updateRenderData(UT_Matrix4T const&, UT_Matrix4T const&, UT_Rect<UT_DimRectImpl> const&, bool) <libHoudiniUSD.so>
tbb::internal::function_task<HUSD_Imaging::launchBackgroundRender(UT_Matrix4T const&, UT_Matrix4T const&, UT_Rect<UT_DimRectImpl> const&, UT_StringRef const&, UT_Options const*, bool)::{lambda()#1}>::execute() <libHoudiniUSD.so>
tbb::internal::custom_schedulertbb::internal::IntelSchedulerTraits::process_bypass_loop(tbb::internal::context_guard_helper&, tbb::task*, long) (custom_scheduler.h:474)
tbb::internal::custom_schedulertbb::internal::IntelSchedulerTraits::local_wait_for_all(tbb::task&, tbb::task*) (custom_scheduler.h:636)
tbb::internal::arena::process(tbb::internal::generic_scheduler&) (arena.cpp:196)
tbb::internal::market::process(rml::job&) (market.cpp:667)
tbb::internal::rml::private_worker::run() (private_server.cpp:266)
tbb::internal::rml::private_worker::thread_routine(void*) (private_server.cpp:219)
start_thread <libpthread.so.0>
__clone <libc.so.6>

Traceback from 6705 ThreadId=0x7f4eb51aaf40
AP_Interface::coreDumpChaser(UTsignalHandlerArg) <libHoudiniUI.so>
AP_Interface::si_CrashHandler::chaser(UTsignalHandlerArg) <libHoudiniUI.so>
signalCallback(UTsignalHandlerArg) <libHoudiniUT.so>
UT_Signal::UT_ComboSignalHandler::operator()(int, siginfo_t*, void*) const <libHoudiniUT.so>
UT_Signal::processSignal(int, siginfo_t*, void*) <libHoudiniUT.so>
_L_unlock_13 <libpthread.so.0>
_GI_ppoll <libc.so.6>
qt_safe_poll(pollfd*, unsigned long, timespec const*) <libQt5Core.so.5>
QEventDispatcherUNIX::processEvents(QFlagsQEventLoop::ProcessEventsFlag) <libQt5Core.so.5>
QXcbNativeInterface::dumpNativeWindows(unsigned long long) const <libQt5XcbQpa.so.5>
UI_QtGeneratorThread::processEvents(QFlagsQEventLoop::ProcessEventsFlag) <libHoudiniUI.so>
UI_QtGeneratorThread::waitForRawEvents() <libHoudiniUI.so>
UI_Queue::waitForGenerators(int) <libHoudiniUI.so>
UI_Queue::waitForEvents() <libHoudiniUI.so>
UI_Queue::generateEvents(bool, bool) <libHoudiniUI.so>
UI_Queue::eventLoop() <libHoudiniUI.so>
main <libHoudiniUI.so>
__libc_start_main <libc.so.6>
main

Error Log.
Unable to successfully run rendering。

from openmoonray.