Instancing about anari-docs HOT 2 CLOSED

Hi,

Instead of responding to why we didn't choose X/Y/Z option, I'll instead give some rationale for what we did design. Then maybe its easier to understand the drawbacks of other ideas.

ANARI is an API to a diversity of rendering engines, all of which may use different techniques for rendering and make different trade offs for raw frame rate (frame latency), scene update rate (animations), or visual quality (materials, GI). We also know that rendering engines are diverse in the algorithms used to generate frames: pure rasterization, hybrid raster/ray tracing, or pure ray tracing.

Why does this matter? Rasterization and ray tracing make different trade offs in what is considered "expensive" to change in a scene. Rasterization is cheap to change the geometry being drawn, but expensive to change the shader(s) controlling appearance. Conversely, ray tracing is more expensive to change the geometry being drawn (BVH rebuilds), but is generally very cheap to change materials. The (geometry + material) --> group --> instance formulation lets backend devices retain flexibility for deciding to sort geometry (ray tracing) or sort shading programs (raster). Given this arrangement of objects, any change to an object can directly express how much internal state must change to accommodate the change (rebuild draw order or refit/rebuild BVH(s)).

It also minimizes redundant calculations: by using groups, we can let an application directly express exactly which objects share a common coordinate system -- exactly the ideal to minimize BVH overhead (both memory and build quality). Of course applications are free to have multiple instances that have the same transform (of different grouped objects), but we can definitively state that the application not expressing common transformations this way is only worse for the device to achieve equivalent performance (bookkeeping or searching for common transform matrices is required to recover this information). Thus we do not expect implementations to go out of their way to reverse-engineer information the application ought to already have -- somebody has to compute what transforms are common, so ANARI assumes the application would optimize for this in their ANARIWorld heirarchy if it is important (some applications may simply not care, such as offline renderering).

So what about multi-level instancing? The current spec is only the minimally required parameters required by an implementation, but we have left the design open to extensions that permit ANARIGroup to contain an array of ANARIInstance, which expresses the nesting of an instance. No ANARI API changes are required for such an extension to be implemented. There is something to be said, though, about the performance of rendering and updating multi-level instanced scenes that is very non-trivial to answer -- multi-level instancing has cost (especially for ray tracers), and almost always has fixed depth limits (Embree and OptiX at least have this), while other APIs simply don't allow it at all (DXR + Vulkan RT) which forces the flattening operation to be opaque and thus unwieldy for applications to intuit its lifetime or overhead. This means that for what seems like a simple feature can actually add lots of entropy to performance intuition to what the cost of using such a feature entails. Again ANARI permits this as an exension, but we did not include it as core due to insufficient clarity that it is better to be underneath the ANARI API instead of being the responsibility of the application's scene graph.

One idea we are wanting to explore is the idea of having transform objects, which would cut down on the number of updates required for multiple instances. Roughly speaking, instances could take a matrix directly (as they do now) or take a handle to a transform, which could exist in more than one instance (set value on the handle once, inherit the value everywhere the transform object was set). This also gives us a chance to have different types of transforms too, such as blurred transforms or even stacked transforms. However, this design was only mentioned as a promising direction, but I am aware of no concrete progress on it yet.

In short, ANARI is trying to be as "thin" as possible between the application and the rendering engine by using a render graph object hierarchy that strives to avoid as much opaque flattening as possible. Given the diversity of implementation choices that rendering engines will make, I don't anticipate multi-level instancing being a required core feature any time soon. However, as soon as an ANARI device implementation successfully implements it as a vendor extension, we would very much consider including it as a core extension to give the feature a consistent design across devices which do support it.

Hope this helps!

Cheers,
Jeff

from anari-docs.

Thanks Jeff. You wrote perfect summary, expressing the difference between rasterization and ray-tracing trade offs. What I see as the most important point for ray-tracing is this using of groups to express a common coordinate system.

You also mentioned Transform objects. I am a big fan of the idea of transform objects. I believe that they could bring great new possibilities. Let me mention some:

1. transformation is shared among attached instances - You already mentioned this. We change one Transform object and all attached instances will get new transformation. The benefit is clear: We might save many updates of transformation matrices.
2. we could have different types of Transforms, such as blurred, as you mentioned
3. hierarchical transformations - we might allow Transform objects to be attached to parent Transform objects and to have attached child Transform objects. You might not want to have this functionality directly in ANARI core, but I see number of benefits of such approach. There might be (a) further reduction of updates of transformation matrices and (b) because we have all the matrices passed through Anari API, we might make all matrix multiplications by ANARI device implementation, possibly using GPU to speed up the computation. It is not much for small scenes, but for millions of objects in non-flat graphs it might make a difference.

When considering the benefits of having Transform objects, we might easily spot that each Transform object serves to provide one local coordinate system that is shared by all attached Instances. In other words, all attached Instances share common coordinate system. I might not see everything, but it might void the benefits of "(geometry + material) --> group --> instance formulation". Without transform objects, I see clearly the intention of (geometry+material)->group->instance and I understand that it was chosen even although there is an overhead of 6 objects allocation for single thing rendered in the scene, e.g. we need 1xInstance, 1xGroup, 1x Array, 1xShape, 1xMaterial and 1xGeometry. For 1000 objects having the same Geometry but different placements and different materials, it is 1000xInstance + 1000xGroup + 1000xArray + 1000xShape + 1000xMaterial + 1xGeometry (5001 handles). I hope, I understood ANARI right. Otherwise, please, correct me.

However, with Transform objects, we might have more efficient design unless I am missing something. If we modify Instance object to contain only reference to Geometry, Material and Transform, we would allocate only 1000 Instances + 1000 Transforms + 1000 Material + 1 Geometry (3001 handles). Each Instance would express its "membership" in particular coordinate system by its attachment to particular Transform object. And each Instance would express its membership in particular group of objects having the same material by its attachment to particular Material object. This way, it would be easy for ray-tracing systems to optimize their BVH rebuilds based on Instance-Transform relation and it would be easy for rasterization systems to optimize for pipeline switches (and pipeline state changes in general) based on Instance-Material relation. Still other system might use even hybrid approaches, so, in some sense, we embrace all of them in a kind of easy to understand way.

Overall, introducing Transform objects and modifying Instance would include these benefits:

• more efficient Instance position updates
• allows for further optimizations like GPU evaluation of large Transform graphs
• possibly lower number of allocated handles (possibly associated additional memory consumption and allocation overhead)
• it would more clearly express the relation between Instance, its coordinate system given by Transform and its rendering state given by Material

One more thought: The current design leads to grouping all scene objects that share the same coordinate system bellow single Instance object. These scene objects might be separated objects and not intended to be considered one instance. But for the sake of ray-tracing performance, we would place them bellow single Instance. I feel like we are misusing the word instance here, using it in the place of coordinate system or something like that.

My goal here is definitely not to criticize. Rather, my goal is ANARI quality before it is released as the final spec. Therefore these thoughts about the efficiency of the scene graph design. Many of my thoughts might turn not relevant, but, maybe, some will contribute to ANARI quality. I am not paid for quality improvements of ANARI, so I cannot do too much at the moment and put my hands fully in, but I hope that ANARI will become important in the future.

from anari-docs.