esri / serlio Goto Github PK

Currently, for Arnold we only support RGBA textures as opacity maps, where the alpha channel is used as the opacity. However, CityEngine also supports RGB opacity maps, where the luminance is used as the opacity.

Task:

• Implement support for RGB opacity maps using the luminance as opacity.

Currently, RPKs are extracted to the system temp directory so renderers (Arnold) can access the textures. This is a problem if generation takes a along time (temp dir might be removed at reboot) or if the project has to be relocated to another machine.

Make the RPK extraction path configurable with the default location within the currently active Maya project.

serlio.mll fails to load at Maya (2019) startup with error message:

// Error: Serlio: the required dependency 'shaderFXPlugin' is not loaded, please activate it! // 
// Error: line 1: initializePlugin function failed (serlio) // 
// Error: line 1: Invalid object or value: // 
// Error: line 1: Invalid object or value: // 

Manually activating the plugin works.

followup of #15

currently, the stingray shader only supports alpha-based opacity maps. add support for rgb-based opacity maps by adding a new shader parameter which tells the shader if the opacitymap has an alpha channel or not and then switch accordingly.

• introduce env var to set the minimally output log level
• remove some of the compile-time guards for debug log messages (careful, some LOG_DBG can be expensive, they are always executed)

Repro:

1. create RPK and generate in serlio
2. recreate RPK by only switching the @StartRule to a different rule
3. bug: after clicking "reload" in serlio the model is still the same

workaround: restart maya

Consider the following simple rule attribute:

@Enum("Val1", "Val2", "Val3", restricted = false)
attr enumVal = "Val1"

CityEngine provides a drop-down menu for this enumeration with entries Val1, Val2 and Val3, but also allows to set an arbitrary value not in this list.

Serlio treats both restricted and non-restricted enums as restricted.

Task:

• Support non-restricted enum attributes, such that a user can set arbitrary values for these attributes

Note: A simple solution would be to ignore the enum annotation on non-restricted enum attributes and simply provide an input field. This way, the user would be able to set an arbitrary value, however, the enumeration options would not be visible in the UI anymore.

define clang-format rules so they are picked up automatically by IDEs like CLion

In CityEngine, a rule attribute can be in two states, namely in default mode or in override mode.

In default mode, the attribute always assumes the default value that is computed by the rule. The default value may change depending on the value of other attributes, the seed or the geometry.

In override mode, the user sets the attribute value explicitly, thereby overriding the default value. The attribute value always assumes the user set value, even if the default value changes when the value of other attributes, the seed or the geometry is altered.

Currently, in serlio all attributes are always treated as if they were in override mode. The attribute default values are computed once when a rule is assigned to a shape and used as the "user set" values for the override mode. A user who wishes to mimic the default mode has to manually compute and set default values if they differ from the default values computed initially. There is also no way to reset an attribute value to its current default value.

Task:

• Implement the behavior of default and override mode for attributes in serlio according to the behavior of CityEngine described above.
• Provide a way for the user to reset attribute values to their default in the UI.

Note:

• Hidden boolean plugs might be a possibility to indicate the attribute modes.
• Issue #49 is not covered by this issue and should probably be fixed first.

Currently, we don't support initial shapes with holes (see screenshot below).

Task:

• Implement support for shapes with holes, i.e. make sure that
  • the initial shape geometry is interpreted correctly (unlike in the screenshot)
  • rules relying on the geometry topology (e.g. rules using deleteHoles) produce the expected result
  • default attributes relying on the geometry topology (e.g. default attributes using the initial shape area) compute the correct value

Current state: Initial shape with a hole (left) having a simple extrusion rule applied (right).

Currently, there's no support for the shininess, reflectivity and bumpValue material properties in either of PRTMaterialNode and ArnoldMaterialNode.

Task:

• Add support for these three properties to the material nodes.

Update Serlio builds to use PRT 2.1

Repro

1. Generate multiple instances of an initial shape randomPlanes.py.zip
   
2. Assign a rule package to the initial shape myPlane1
   
3. Select the initial shape and Serlio > Create Materials
   BUG: The Materials are not attached to the instanced mesh.
   

For our material shaders (both Arnold and viewport materials), we currently use a fragile approach of marking our nodes as sequential (so that their compute method doesn't get executed in parallel to other node's compute method) as well as sequentially executing MEL scripts to ensure that the shader names of our material shaders are unique. This is not necessary, since Maya itself already ensures uniqueness of node names.

Task:

• Adapt our material nodes to rely on Maya's unique node name assignment instead of trying to ensure uniqueness ourselves.

e.g. the "Height" attribute is missing the slider despite of having a correct "@range" annotation in the CGA rule

Update the "Favela" example with Arnold renders
... or ...
provide a new example which takes advantage of Arnold

Currently, when setting up the Arnold material shaders for a model, we don't consider already existing and matching material shader from other models and instantiate a new material shader in any case. Even worse, when changing model attributes and regenerating the model and materials, the material shaders from before the change are ignored and all material shaders are newly created.

Task:

• Implement material shader reuse across all models and model regenerations.

Note: for the viewport material we already have deduplication

Tasks:

• Officially add support for Maya 2020 in the build system
• Provide binaries for the next release

the stingray shader unexpectedly multiplies rgb-based opacity with alpha-based opacity. CityEngine falls back to rgb-based opacity if there is no alpha channel in the texture. serlio should emulate that CityEngine behavior.

reproduction: load CityEngine Paris example in Serlio

Trying to manually load serlio.mll from plugin manager gives this error:
// Error: file: C:/Program Files/Autodesk/Maya2022/scripts/others/pluginWin.mel line 316: The specified module could not be found.
(serlio) //

Repro

1. Create a Serlio Node
   WindTurbine_Esri.zip
2. Serlio > Create Arnold Materials
3. Set Key Frame or use keyRotation.py.zip
4. Set another Key Frame
   BUG: Infinite creation of material nodes
   

We currently persist data structures of type adsk::Data::Structure (accessed via mesh stream channels) in the scene. These structures are created by the PRT node and consumed by the Stringray/Arnold material nodes to create the MEL scripts necessary to actually create the materials.

Why do we persist these?

• Running Serlio on a shape with disabled construction history will only store the result (there is no Serlio node), i.e. store the material data structure is the only way to later actually create the material.

There are several issues with persisting the current material "proto" data structures:

• These data structures are not versioned, it is not possible to do a clean legacy handling in future Serlio releases in case the structure changes.
• The structures are dynamically derived from the generated PRT materials, i.e. a black box from Serlio pov. Changes in PRT could in-transparently lead to different material structures and crashes when Serlio tries to read it.
• Minor: we currently convert the original wide-character string attributes to ASCII when storing in the material data structure. This will lead to problems if e.g. texture strings contain non-ascii characters. Also it requires a lot of redundant string conversions. See branch simo6772/cleanup-material-structure for preliminary work on this.

Tasks:

1. Investigate if it is possible to avoid persisting the material data structures and therefore the need for versioning. Unfortunately, it is very likely that using Serlio with disabled construction history requires persisting the material structure nonetheless.
2. Investigate if we actually need to support construction history turned off, i.e. is it a supported case that a Maya plugin only supports construction history turned on?
3. Based on investigations, write implementation tickets to improve the above issues.

Reproduction:

1. Assign RPK
2. Keyframe an attribute
3. Copy/Paste the model -> crash

Currently, in Arnold we ignore the opacity map mode (material.opacitymap.mode can be blend, mask or opaque) and simply use the default blend.

Task:

• Implement support for the mask and opaque mode of opacity maps.

Observed in Maya 2019 (Arnold 5.2):

1. Create RPK with rule:

   Init --> i("/ESRI.lib/assets/Webstyles/Vegetation/Realistic/AbiesBalsamea.glb")
   

2. Assign the RPK to a quad in a fresh maya scene
3. Add an Arnold light
4. (x) note how the leaf cards are not correctly cutout (no sprite rendering)
5. Workaorund: manually disable the "Opaque" checkbox at pPlaneShape1 -> Arnold

We currently redirect logging output from PRT to the std out which appears in different places on windows and linux.

Use the MGlobal::display{info, warning, error} functions instead.

For enum attributes, the default value is always set to the first value listed in the enumeration, ignoring the actually set default.

The following rule erroneously defaults to "Val1":

@Enum("Val1", "Val2", "Val3")
attr testVal = "Val2"

Task

• Make enum attributes default to the right default value.

Task
After assigning a rule package, loop over all attributes of the initial shape and connect matching attributes with the PRT node.
example:

// Connect the height attribute of the initial shape nodes with the PRT node
connectAttr polyPlane1.PRTDefault_height serlio1.PRTDefault_height;

Both the PRTMaterialNode and the ArnoldMaterialNode need to find mesh nodes downstream the dependency graph. Currently, this is manually implemented, only following the plug connection chain via groupParts nodes (as these auxiliary nodes are inserted, when materials are assigned on the face level).

Task:

• Reimplement this search logic properly and with support for intermediate nodes other than groupParts nodes. MItDependencyGraph is likely to help.

In a newly created Maya scene, there are no light sources added by default. Therefore, we always add a physical sky light source to the scene, when we attach Arnold materials to a PRT model, since without a light source Arnold just renders black. However, if there are already existing light sources in the scene, this might not be desirable.

Task:

• Decide on a reasonable strategy for automatically adding light sources and implement it.

Currently, we don't support rotation of UV coordinates in Arnold materials, since the order in which scale, translation and rotation is applied in CityEngine seems to be different than the order applied in the aiUvTransform shaders used in Arnold materials.

Task:

• Implement support for rotation of UV coordinates.

Ideas:

• aiUvTransform could be replace by or combined with place2dtexture
• chain multiple aiUvTransform to enforce the order in which scale, translation and rotation is applied

Currently, in both PRTMaterialNode and ArnoldMaterialNode we use a separate texture shaders for each map, regardless of whether the textures are actually the same or different.

Task:

• Create only one texture shader for different maps using the same texture file.

(Do we need to support the case where a single texture file is used in multiple maps with different UV transforms?)

• ensures same source state for each step
• less requests to github

Instancing of a shape in Maya is lightweight shape duplication. Instead of copying the shape geometry for each duplicate, the geometry is simply appended as a child to multiple different transform nodes that position each duplicate (i.e. the shape has multiple parent transform nodes).

Once we use world space vertex coordinates to build the initial shape (Issue #35), we need to think about how we handle instanced Serlio nodes, since the initial shape geometry has multiple world space vertex coordinates (one for each parent transform).

Task:

• Define and implement a strategy for determining the world space vertex coordinates of instanced Serlio nodes.

When generating Arnold material for multiple models, we always reuse the same variables in the emitted MEL script, which leads to a variable shadowing warning.

Task:

• Find a way to avoid these warnings.

Note: One possibility we might try is to encapsulate the MEL script logic in a command using C++, which as a bonus might even execute faster than the MEL script.

Tasks:

• register an ID block with Autodesk (see http://mayaid.herokuapp.com/)
• use the ID block to assign node IDs to all nodes (currently PRTModifierNode, PRTMaterialNode and ArnoldMaterialNode) defined by the plugin (see MTypeId doc)

Repro

1. Create an RPK using @enum attributes
   @Enum( "Airplane", "Box", "Train" )
attr asset = "Train"
2. Assign the Rule Package insertAsset.zip in Maya
3. Select asset "Airplane"
   BUG: The default asset "Train" is loaded.
   

In Maya, shape vertex coordinates are stored in a local coordinate system and a transform node hierarchy is used to position the shapes in the scene, thus the world space vertex coordinates of a shape are obtained by applying the transformations in the node hierarchy to the local vertex coordinates. Currently, we only use the local vertex coordinates of a shape to build the initial shape, which differs from the behavior in CityEngine, where the world space vertex coordinates are used.

Task:

• Adapt Serlio to use world space vertex coordinates when building the initial shape.

The setup of the material structure is currently done lazily in MayaCallbacks::addMesh. This is not clean and deregistering the structure is missing as well. Refactor this.

Currently, in both PRTMaterialNode and ArnoldMaterialNode finding the shading group corresponding to a material shader is implemented by simply changing a known infix in the node name. E.g. a shader node with the name serlioGeneratedArnoldMaterialSh7 will match to the shading group with the name serlioGeneratedArnoldMaterialSg7. However, this method is unreliable, since nodes can be renamed by the user.

Task:

• Find a more reliable way to match a shader with its shading group, e.g. by inspecting the attribute connections, since the .outColor attribute of a shader is connected to the .surfaceShader attribute of the corresponding shading group.

Repro

1. Mark an attribute as a file name.
   Example:

@File
attr test0300File = "myfile.obj"

2. The Serlio inspector will present a file chooser.

In CityEngine, we compute a random seed for a shape once based on its geometry. After that, the user can freely change the seed, but there is no way to reset the seed to its initial value implicitly (i.e. you have to explicitly remember the initial seed and manually set it to achieve a reset).

Essentially, the seed is an attribute that is always in override mode (as described in #50) and initialized to a random seed.

In serlio, we are not consistent, but use different seeds to compute the default attribute values and generate models. As far as I could see, the random seed for generating the models even relies on uninitialized memory (i.e. it just uses whatever value is returned from an uninitialized plug).

Task:

• Ensure that the seeds for both default attribute computation and model generation are always the same.
• Make the behavior of the seed consistent with CityEngine.

Currently, Serlio uses the rule with "@StartRule" annotation as the start rule. This is not sufficient for complex RPKs. E.g. some RPKs could contain building and street rules.

Tasks:

1. Add a dropdown attribute below "Rule Package" in the "CGA Rules" category of the serlio editor node
2. Populate the dropdown with all rules without parameters from the rule file (cgb) which contains the "@StartRule" annotation.
3. If possible within Maya API, visually emphasize the rule in the dropdown which is annotated with "@StartRule"
4. Ensure the dropdown updates correctly when switching and reloading RPKs.

In the Stringray material node, repeat the same code cleanups and potentially share code from the Arnold material node.

If the menu item "Attach rule..." is applied on an already generated PRT model, Serlio should switch the RPK, not apply the rule on the generated model (which likely results in huge resource consumption and brings maya to standstill).

Implement material creation for Arnold, so that PRT models render out of the box (in viewport and in Arnold render view).

Open Questions:

• investigate best approach: second PRT material node or a "switch" on the current material node?
• should we use the same approach as for stingray, i.e. one arnold material node per PRT material OR adding a custom arnold shader/material which can read the shader properties directly from the mesh metadata (= less materials in maya, better performance)?