Render Multiple Cameras at a given Time about com.unity.perception HOT 16 CLOSED

Hi there, I'm glad you found a workaround. Much of the Perception package has been designed with multiple cameras in mind, but we still have a few issues to work out. Your approach of using the Camera's instance ID is a good one.

And yes, you will need to assign RenderTextures to each camera, at least making sure that no more than one camera is rendering to the same Display.

No guarantees, but we will try to get official support in for the next release, which should be around the first week of September.

from com.unity.perception.

Thanks everyone for your feedback on this. Multiple simultaneous camera support has been added in the latest 0.9.0 release of the package. Try it out and let us know if you run into more issues.

from com.unity.perception.

Sorry I have found it and now include the instance ID in the filename, however when it comes to using the BoundingBoxLabeler, in the JSON files produces the filename does not reflect this change. Was this an issue you also encountered?
Thanks for the quick response.

from com.unity.perception.

NOTE: Setting up multiple RenderTexture assets in the project and assigning unique RenderTextures to each Camera component seems to solve our issues.

Otherwise, it seems they might use the same underlying frame buffer and overwrite each other.

from com.unity.perception.

Ooh, this is amazing news, thanks so much for the fast & detailed response!

Goodluck with implementing it :).

from com.unity.perception.

Ahh silly me, just an extra note that you're probably aware of:

When my PerceptionCameras used semantic segmentation, support would need to include a fix for this error:

Hope this helps!

from com.unity.perception.

I am also having the same problem. Could you tell me what you edited in PerceptionCamera using the InstanceID?

from com.unity.perception.

@jamesheatonrdm Ah, I can't remember and no longer have access to the repo where we made the change.

However, if you have the Perception package source, see PerceptionCamera.cs and try to find the file/folder path it does use, and change it there using GetInstanceID().

Apologies for the vague answer!

from com.unity.perception.

I do now see that the json files include the sensor id with the filename for each camera, so I can use that to determine which camera is producing which images and their bounding boxes.

from com.unity.perception.

You're welcome! Huh, interesting.. it does sound familiar.

Is it possible to see how it produces this JSON file, to ensure it's referencing the proper file name? 🤔
I unfortunately don't have any specifics though to help. Goodluck though, you're definitely on the right track!

from com.unity.perception.

When using using Render Textures to output and write ground truth images from multiple Camera at once, the images produced seem to be very dim compared to outputing to different Displays?

Using render textures

Using displays and no render texture

from com.unity.perception.

WeiKin,
I'm trying to pin down why this is happening, and am curious about the colorspace you are using. The darkness looks possibly like the difference between Linear and Gamma, however a Render Texture should follow the colorspace of the project. I have a few questions to try to pin this down:

1. What colorspace is your project using? You can find this under Edit > Project Settings > Player > Other Settings
2. How are you creating the Render Textures, in Editor or via code?
3. Do your Render Textures (if you can view them in the Editor) match the colorspace of the project?
   
4. In creating the Render Texture, have you specified the color format?
5. Do all of your cameras share the same settings for ISO, shutter speed, aperture, etc.?
6. Are you using any Tone Mapping in post processing that would affect a camera using the Display versus a Render Texture?

Sincerely,
Adam

from com.unity.perception.

• The render texture looks the same as what you have with the color space of the project and texture both being Linear.
• I am creating the Render Textures in the Editor, and only changing the size and nothing else.
• The cameras are all duplicated and have their outputs just changed to fit a new render texture for each camera, so they all have the same settings
• No I don't think I have added any post processing other than whatever is default in the HDRP project settings

from com.unity.perception.

Thanks for your updating new features. It saves my life. However, after import perception library with 0.9.0-preview.2 version. An error says

Library/PackageCache/[email protected]/Runtime/Randomization/Samplers/ISampler.cs(40,22): error CS0106: The modifier 'public' is not valid for this item

Then, I tried to remove the modifier 'public'. But it recovered after my unity compiled the code. So, I comment all line with modifier in the interface ISampler, then It works. However, I guess this may lose the functionality of the random sampler.

from com.unity.perception.

Thanks for the report. We are looking into it.

from com.unity.perception.

The latest package supports only 2020.3.17f1 and beyond. Please update to this version of Unity to fix the compilation error @harry83017622

from com.unity.perception.