How do I convert Unity image annotations into annotations usable for training an ML model? about com.unity.perception HOT 6 CLOSED

You can use the converter tool I have published here:
https://github.com/lessw2020/perception_tools

There is both a notebook and a script. For the notebook run all the cells down to the begin processing.
Set the path to the perception directory, adjust the image height and width and the output name if needed...and run.
Will process and produce a training ready Coco format file for you.

Note that this is for 2D bounding boxes. I have not implemented segmentation yet.

from com.unity.perception.

Wonderful @lessw2020!
For other types of models, we do provide the library Dataset Insights, a python library for loading Perception-based datasets. This should make it straightforward to feed the data into your model training code.

from com.unity.perception.

Firs of all, THANK YOU!
Second, I am running on Linux, getting this error consistently:
"directory of type Perception not found. aborting..."

In the instructions you refer to the "perception directory". What do you mean by that?

from com.unity.perception.

Hi @Tylersuard:
Definitely welcome.
'perception directory' should be the folder where Unity/perception puts it's three main output folders - Dataset*, Logs, and RGB*.
The error is triggered if that path is not of type directory. (are you passing in a path to the file instead of the directory?)

Example from my drive (though this is windows):

Thus the 'perception_directory' is the full path to that highlighted dir, 26cd1c73-4631-4b67-bfb1-9726d23d8de6.

From there, the converter will look for the first Dataset* folder and use that to get the annotations, and could also use the RGB folder if needed to check images/sizes, etc.

It should work fine on Linux, so if you can verify you are passing in path to the directory that should let you proceed.
If not, feel free to just post a screenshot of what you are passing in, and if can also update it to strip a filename and get the parent dir.

from com.unity.perception.

did anyone try to use these annotations and register them into detectron2 models ?
can we use directly like this
https://detectron2.readthedocs.io/en/latest/tutorials/datasets.html#:~:text=from%20detectron2.data.datasets%20import%20register_coco_instances%0Aregister_coco_instances(%22my_dataset%22%2C%20%7B%7D%2C%20%22json_annotation.json%22%2C%20%22path/to/image/dir%22)

from detectron2.data.datasets import register_coco_instances
register_coco_instances("my_dataset", {}, "json_annotation.json", "path/to/image/dir")

thank you @lessw2020 for providing your code, ill try it soon !

from com.unity.perception.

Hi @orangeRobot990 - I didn't use synthetic inside of detectron2, but have worked with detectron2 beyond that, and would say it should work fine to import the synthetic with coco annotations.
Once the synthetic images are annotated with coco format like I setup, they will work anywhere coco labelling is accepted.

Hope that helps!
Less

from com.unity.perception.