The build fails with the latest USD version, due to the removal of UsdShadeMaterial::GetBoundMaterial.

To Reproduce
Steps to reproduce the behavior:

1. Build the procedural against the latest cut of USD.

Used Software Versions

• USD: [e.g. 0.20.2]

We should handle arnold metadata and store additional information in the ndr plugins.

This include:

• metadata on arnold nodes
• metadata on arnold parameters
• metadata storing the arnold type of the parameter (there are arnold types that map to the same sdf type)

Describe the solution you'd like
We need to extend ndr/utils.cpp to deal with metadata. Add additional USD metadata to store the raw arnold type on the primitives in the Usd Stage returned by NdrArnoldGetShaderDefs. And pass metadata to Usd in the Discovery and Parser plugins.

Add a usdKatana scenegraph location decorator plugin that allows converting Arnold specific attributes to KtoA compatible attributes. This also includes supporting remapping of namespaced primvars to built-in Arnold parameters, a feature that is supported by the Render Delegate.

example:
usd-arnold Katana plugin

Describe the bug
Currently, our schemas generation does some exceptions for a few attributes, whose name (or values) causes problems for USD. This happens because usdGenSchema creates a file tokens.h with all the attribute names and enum tokens, and compiles it. But some of the names can't compile.

Here's the current list :
//////////////////////////////////////////////

The attribute namespace (from procedural), will introduce this in tokens.h :

/// UsdArnoldAlembic
const TfToken nameprefix;
/// \brief "namespace"

and then fails to compile :

tokens.h(2992): error C2059: syntax error: 'namespace'

The attribute default (from user_data_* ) seems to be a problem for usdGenSchema as it adds an underscore :

/// 
/// UsdArnoldUserDataString, UsdArnoldUserDataRgb, UsdArnoldUserDataRgba, UsdArnoldUserDataInt, UsdArnoldUserDataFloat
const TfToken default_;

and then fails to compile :

build\windows_x86_64\msvc_opt\usd-0.19.5_arnold-6.0.1.0\schemas\source\arnoldUse
rDataRgba.cpp(125): error C2039: 'default': is not a member of 'arnoldPxrInterna
l__pxrReserved__::UsdArnoldTokensType'

We have a similar problem with enum values, since they're also added as tokens in this same file.

Value auto (from polymesh.subdiv_adaptive_metric) creates this:

/// \brief "auto"
/// 
/// Possible value for UsdArnoldPolymeshAPI::GetArnoldSubdiv_adaptive_metricAttr(), Default value for UsdArnoldPolymeshAPI::GetArnoldSubdiv_adaptive_metricAttr(), Possible value for UsdArnoldPolymesh::GetSubdiv_adaptive_metricAttr(), Default value for UsdArnoldPolymesh::GetSubdiv_adaptive_metricAttr()
const TfToken auto;

which obviously fails to compile

0\schemas\source\.//tokens.h(684): error C2628: 'arnoldPxrInternal__pxrReserved_
_::TfToken' followed by 'auto' is illegal (did you forget a ';'?)

Our shader attribute compare.test has enum values like ==, <, >, etc...
and this is a problem for usdGenSchema:

ERROR: string index out of range

Finally, we have problems with tokens upper/lower casing. Arnold's utility shader has a parameter color_mode with possible values p, n, ng, ns... but state_vector.Ro has values P, N, Ng, Ns.... This isn't allowed in schema tokens, and fails with :

ERROR: Token identifiers must map to exactly one token value. One-to-Many mappin
g encountered: ng maps to "ng" and "Ng"

The Ndr Discovery and Parser plugins should register all the other node types available in arnold.

Describe the solution you'd like
All we need to do is to change the Node Entry iterator and manually handle the options node in ndr/utils.h.

Document naming conventions for the various Arnold entities, via a markdown file and add it to doxygen.

• Properties
• Attributes
• Primvars
• Schemas
• Namespaces

Describe the bug
When I open the PointInstancedMedCity example file with usdView and choose Arnold as the renderer, then the whole usdView app crashes. It renders nicely with OGL and Embree.

To Reproduce

1. Open usdView with the PointInstancedMedCity.usd
2. Select Arnold from the Hydra renderers.
3. Watch usdView disappear.

Used Software Versions

• Arnold: 5.4.0.2
• USD: 19.11
• Compiler: MSVC 2019
• OS: Win 10

The Procedural supports using a static USD build, but that limits extensibility and the option to run custom plugins. We need to investigate what's the best option to load plugins in a procedural that's using a static build of USD.

This most likely requires shipping the USD headers, pluginInfos, and exposing the USD symbols in the procedural.

At the moment the Render Delegate is using the P AOV to calculate the OpenGL depth and the ID AOV to differentiate between the background and foreground pixels.

The Z AOV should be better suited for both problems, as the P and ID AOV's behavior is inconsistent when using the GPU backend or mixing volumes and other objects.

Arnold encounters error when loading a usdz file that has embedded textures.

To Reproduce

1. Download usd test file simpleMaterial.usdz
2. Run: usdview --renderer Arnold simpleMaterial.usdz
3. Get error ERROR | [texturesys] Image "C:\Users\treborc\Documents\usdme\pxr\usd\lib\usdUtils\testenv\testUsdUtilsCreateNewUsdzPackage\nestedUsdz\simpleMaterial.usdz[0/USDLogoLrg.png]" does not exist. Also, it is not the name of an image format that OpenImageIO recognizes.

Expected behavior
Able to open the usdz file without errors and load texture

Used Software Versions

• Arnold: [5.4.0.0]
• USD: [0.19.11]
• Compiler: [MSVC 2017]
• OS: [Windows 10]

On linux, running usdview with the arnold render delegate crashes. The current workaround for this is to ld-preload the arnold library.
So we need to run

LD_PRELOAD=/path_to_arnold/bin/libai.so usdview scene.usda

and this prevents the crashes

Core ticket: #8869

The build system requires setting up the USD_HAS_PYTHON_SUPPORT to know if USD was built with python support or not. This is tracked in the installed headers, so we should be able to check for this during parsing the USD version.

Neither the Procedural nor the Translators make use of the standard TfDebug toolset, which allows fine-grained control over the printed debug messages. Adding debug output would greatly simplify any future debugging.

The material adapter and the Render Delegate debug codes are good examples to follow.

maketx is part of the contrib folder, but it's not used for anything. Just clean it. If we need it in the future, the Arnold SDK does ship one.

The houdini python library has a -mt suffix on windows.

Describe the bug
The #include <ai_map.h> in arnold-usd\translator\reader\registry.h fails when compiling against Arnold 5. I assume ai_map is only part of Arnold 6.

To Reproduce
Steps to reproduce the behavior:
Just try compiling.

Used Software Versions

• Arnold: 5.4.0.2
• USD: 0.19.11]
• Compiler: MSVC 2019
• OS: Win 10

Additional context
Add any other context about the problem here.

Refactor the render buffer system:

• Write to the render buffers directly and avoid intermediate storage.
• Support custom AOVs with the type color3f.
• Support LPEs.
• Optimize writing to buffers to improve interactivity.

HdEmbree is a good example

Support the Hydra point instancer for primitives.

Initial support was already added for instancing meshes see render_delegate/instancer.cpp and related files.

A few features are still missing:

• add support for instancing volumes
• calculating transformation motion blur for instance matrices

We can use hdEmbree/instancer.cpp as an example implementation.

Add tests for the Render Delegate.

We can use usdrecord from the newer USD versions to render any USD scene via an offscreen buffer. The result can be compared to a reference image, similarly to the current tests for the Procedural.

Katana 3.2 introduced support for Hydra render delegates, and we should make sure the render delegate works with Katana 3.2 and later releases.

Newer USD expanded the HdxCompositor to allow for non-8 bit int buffers for the color. We should take advantage of this and use f32 for beauty.

Related to: #8

There should be a render setting to turn on the Optix denoiser for the beauty AOV.

The difficulty here is that the composition, responsible for differentiating between background and foreground pixels, happens at the moment per bucket, and the Optix denoised beauty is available at a different rate than the non-denoised ID and P AOV. We need to remove the requirement of having all 3 buffers available at the same time for each bucket and update how composition behaves.

UsdMaterialBindingAPI supports the use of material binding caches, which make computing material bindings more efficient.

Describe the solution you'd like
The procedural should use a material binding cache to compute its material bindings.

Additional context
This ticket is the continuation of #59 .

Using Hydra and UsdImaging for a procedural is an alternative approach to directly accessing and traversing a Usd Stage. This is a similar approach to what the HdPrman plugin does and it offers significant code reuse between the Render Delegate and the Procedural. Besides this, Hydra also takes care of parallel access to the stage automatically.

Creating a procedural prototype should be straightforward, but the most important thing is to make sure performance is similar to a handwritten procedural

HdPrman example

Hi !

Amazing to see this repo go live !
I've been successfully able to compile against Houdini's shipped libraries with a few modifications and was wondering if these could become part of the main build system.

First, USD libs in Houdini have the libpxr_ prefix, so it would be nice to have a build option like BOOST_LIB_NAME for USD.
Second, it seems like the USD_HAS_UPDATED_COMPOSITOR doesn't set correctly when building against Houdini's USD libs. Setting it up manually has fixed it in my experience.

Thanks !

Let's move the husdshadertranslators python module, and the HdArnoldRendererPlugin*.ds files to HtoA, and so remove them from the arnold_usd repository.

Describe the feature you'd like

At this stage it's unclear how to actually use the arnold-usd repository and how to adopt or even test the functionality.

Describe the solution you'd like

1. It would be good to briefly elaborate what the correct environment should be after building, e.g. what should be added to PATH or maybe to some Arnold-related environment variables to have the procedurals/plug-ins found. It would be good to have a simple example (potentially per OS).

2. With arnold_to_usd as a helper utility it could be good to have a Getting Started tutorial that describes taking an example Arnold .ass file and take it through the executable.

For example:

1. Set up your environment

:: Psuedocode (random example)
set PATH={root}/bin;%PATH%
set PATH={root}/lib;%PATH%
set PATH={root}/procedural;%PATH%
set ARNOLD_PLUGIN_PATH={root}/plugin

2. Download an example .ass file from Arnoldpedia

3. Run arnold_to_usd to convert it to a usd stage.

arnold_to_usd /path/to/input.ass /path/to/output.usd

4. Add the procedural to an Arnold scene and load the created USD scene. (or can we render it directly?)

5. (Optional) Show how one could layer a change into the .usd file and influence the Arnold render using a secondary .usd file.

#usda 1.0

over "Camera"
{
    float[] arnold:focus_distance = [50.0]
}

Describe the feature you'd like
The ability to specify a full path to the gcc compiler, rather than restricting it to gcc only

Describe the solution you'd like
Able to set COMPILER="/path/to/bin/gcc in the custom.py file

Describe alternatives you've considered
Tried to set it, but ALLOWED_COMPILERS does not permit this. Also tried setting CXX and CC in the custom.py file with no luck

Luma Pictures' usd-arnold provides a Maya Shading Mode Exporter plugin, which can be used to export Arnold shader definitions from usdMaya. Even though this repository might not be the final place for that exporter, we should allow early adopters to build the plugin and use it in its current form.

The usdMaya plugin

Currently the USD writer only converts Arnold to arnold-specific schemas (ArnoldStandardSurface, ArnoldPolymesh, ArnoldSkydomeLight, etc....).

We should allow, at least optionally if not by default, to export to USD-native schemas when it makes sense.
For example an skydome_light could exported as a Dome primitive, and we'd add the additional arnold attributes in the arnold: scope.

Currently, the functions for pushing primvar data to Arnold nodes will return early if a user parameter cannot be declared. This prevents dirty primvars from propagating to Arnold after they are initially declared.

Currently, performing a blocking Hydra render to completion requires a pattern similar to the following:

auto tasks = taskController->GetRenderingTasks();
do {
    engine.Execute(renderIndex, &tasks);
}
while (!taskController->IsConverged());

Right now, if the render performed by HdArnoldRenderPass aborts for any reason, the render pass is not marked as converged, and thus the above code will block indefinitely.

I think the most "correct" thing to do in the case of an Arnold render error is to mark the pass as converged and zero the render pass' output buffers.

To make the render delegate/procedural more flexible for users and allow them to modify its behavior, we should support writing plugins for the render delegate/procedural.

There are several ways we could allow users to customize behavior.

1, define custom material remapping functions, similar to the ones we use in the render delegate. The interface is already well defined, we just have to expose the material context class and provide a few example callbacks.

For example, someone could use this to translate a custom set of OpenGL shaders to Arnold shaders.

2, custom USD primitive to Arnold node writer for the procedural. The procedural works by defining writers for each supported USD primitive type, we can expose this base class and allow rendering of custom primitives this way.

For example, Luma has a custom primitive type for image planes in USD, and this interface should allow them to convert that to a background image.

3, custom Hydra to Arnold node. The render delegate creates Hd classes to translate between the Hydra primitives and the Arnold nodes, we should expose a plugin registry that allows users to register their own classes for custom Hydra types.

The USD ecosystem has a so-called plugin registry system. There are utility macros that let everybody easily define a custom plugin system for any sort of functionality. A good example is the Maya shading mode system.

Example classes:

The shading mode registry

This deals with registering and returning registered shading mode writers. Just a simple singleton class, that uses the TfType system to call into dynamic libraries that can register their own writer functions.

The display color shading mode

The following code showcases how to register a new shading mode writer.

TF_REGISTRY_FUNCTION_WITH_TAG(UsdMayaShadingModeExportContext, displayColor)
{
    UsdMayaShadingModeRegistry::GetInstance().RegisterExporter(
        "displayColor",
        []() -> UsdMayaShadingModeExporterPtr {
            return UsdMayaShadingModeExporterPtr(
                static_cast<UsdMayaShadingModeExporter*>(
                    new DisplayColorShadingModeExporter()));
        }
    );
}

The classes are already set up for external use in the render delegate, and headers are installed as part of the package. The same is not yet done for the procedural.

Add support for the Hydra camera primitives and the physical camera attributes.

References:
PixarAnimationStudios/OpenUSD@c77e2e4
PixarAnimationStudios/OpenUSD@0b84f3a

Add support for reading the vertex normals from the Scene Delegate. This feature is useful if the polymesh is not subdivided and the user supplies custom vertex normals.

This requires adding HdChangeTracker::DirtyNormals to the initial dirty bits and query HdTokens->normals from the scene delegate.

A good example is : hdEmbree/mesh.cpp

Add support for hydra curves.

We can use hdPrman/basisCurves.cpp as an example implementation.

USD allows renderers to define shader and node registry plugins. This lets users and different tools to access node/shader parameter types and default values using USD tools.

Existing implementation from Luma Pictures

In Arnold 6, procedurals can define a new function procedural_viewport that determines what is displayed in the viewport. This ticket is about implementing this function in an efficient way, by just loading what is needed

USD allows writing powerful custom file format plugins. An ass plugin would allow us to do the following:

• reference ass files in USD files
• open ass files via usdview directly and use OpenGL or Arnold to preview
• write ass files from a USD stage
• convert between ass and usd using usdcat

The feature is best implemented following the example of the
USD Alembic example plugin .

We should add a set of Schema APIs that simplify using USD stages in an Arnold focused pipeline. Due to USD's great python support, this would allow us to give TDs and Developers a great C++/Python toolset that's easy to use.

Some of the features we could add:

• Adding Arnold attributes to shapes
• Creating special primvars for Arnold built-in attributes
• Creating and setting visibility/sidedness/autobump flags easily
• Setting up Arnold shading terminals

Example:
usd-arnold's UsdAi API

The procedural checks if a mesh has subdivision by invoking the function UsdGeomMesh::GetSubdivisionSchemeAttr() , which returns catmull-clark by default.

The problem is that Arnold has polymesh.subdiv_iteration=1 by default. The combination of these 2 defaults causes all meshes to always appear with a 1-level catmull-clark subdivision when rendered with the procedural, which should be changed

The Render Delegate should utilize the TfDebug utilities and print information about receiving invalid data and extend it's currently used debug codes.

The material adapter already makes use of TfDebug, and we should follow its example.

Add tests for the Writer command.

A good approach is to execute python scripts as part of the testsuite and compare the resulting USD file to a reference file.

Add support for Hydra's coordSys feature that allows shaders to define coordinate systems for transformations.

OpenUSD page

GitHub allows the use of templates for issues and pull-requests, this allows setting the value of the description box (and others) to guide the users in creating issues. This is done via storing markdown files in the .github subfolder of the repository.

GitHub Documentation

USD repository examples

The default smoothing follows the Arnold default (off). We should invert by default like in all plugins.

Primvars are usually converted to arnold user data.
But as it is currently done in the render delegate, primvars in the arnold namespace, e.g. "primvars:arnold:max_bounces" should be converted to arnold builtin attributes.

This behaviour is currently helpful in the render delegate for the DCC / plugins integration, so we should mirror it in the procedural for consistency.

The render delegate does not support per face material assignments at this time. We need to investigate what are the best options to do so.

Currently the objects happen to compute a matrix obtained by calling UsdGeomXformable::GetLocalTransformation , since there's no function in UsdGeomXformable to get a world matrix. Objects are obviously not in the right position if their parent was transformed.

In order to get the right matrix, we should use a XformCache that efficiently caches the hierarchy matrices for multiple uses.

Describe the bug

Compiling fails to build against USD 19.11 with this error:

render_delegate\render_pass.cpp(177): error C2660: 'pxrInternal_v0_19__pxrReserved__::HdxCompositor::UpdateColor': function does not take 3 arguments
D:\dev\usd\build\include\pxr/imaging/hdx/compositor.h(66): note: see declaration of 'pxrInternal_v0_19__pxrReserved__::HdxCompositor::UpdateColor'

The built usd version is: 0.19.11. Which according to the merge release of 19.11 contains the updated UpdateColor arguments including HdFormat format.

As such this falsely enters the else statement due to USD_HAS_UPDATED_COMPOSITOR only being defined for USD 19.12 and up.

To Reproduce
Steps to reproduce the behavior:

1. Compile USD 19.11
2. Compile arnold-usd against it.
3. Error.

Expected behavior
It should compile correctly.

It continues compiling when this line is changed to:

#if USED_USD_VERSION_GREATER_EQ(19, 11)

Used Software Versions

• Arnold: 5.4.0.2
• USD: `0.19.11
• Compiler: Visual Studio 2017
• OS: Windows 10