allenai / procthor Goto Github PK

View Code? Open in Web Editor NEW

251.0 9.0 20.0 4.06 MB

🏘️ Scaling Embodied AI by Procedurally Generating Interactive 3D Houses

Home Page: https://procthor.allenai.org/

License: Apache License 2.0

Python 100.00%

ai2-thor computer-vision embodied-ai procedural-generation robotics

procthor's Issues

reachable positions in room

Dear author,
Is there a way to know the boundaries between rooms, such as the reachable positions in the kitchen and reachable positions in the bedroom?
Thanks,
Juan

Hi authors, thanks for the awesome work! I noticed that in Objaverse paper you mentioned there are 10K homes generated with ProcTHOR populated with objects from OBJAVERSE-LVIS. I wonder if the scripts of integrating ProcTHOR and Objaverse can be provided? Or is it possible to provide the scene json/unity file?

Multiagent error?

import prior 
from ai2thor.controller import Controller

dataset = prior.load_dataset("procthor-10k")
house = dataset['train'][0]

c = Controller(scene=house, height=640, width=640, fieldOfView=90, agentCount=2)

#c.reset(agentCount=5) => not work
#e = c.step(action="Initialize",raise_for_failure=True,**c.initialization_parameters,) => not work

Only a agent in the scene.

+edit

I noticed that there is an if statement on line 301 of the AgentManager.cs script in the AI2-THOR GitHub repository (https://github.com/allenai/ai2thor/blob/63947e75d0762101a141d079a4a2737f7a4656f3/unity/Assets/Scripts/AgentManager.cs#L301). If I were to passively delete this statement, would it cause any issues?

Thank you for your assistance.

[Request] Generation is Slow of ProcTHOR Houses at Scale

Description

Hi Folks,

I am trying to use ProcTHOR house generation during DD-PPO training and I found the house generation is quite slow at the moment. I did some preliminary profiling and found these methods are the bottleneck for slow generation:

house_generator -> add_small_objects (48% of total generation time):

1. controller step - spawn asset (12% of total time)
2. controller step - reset(7.8% of total time)
3. controller step - initial random spawn (7.8% of total time)
4. controller step - reset object filter (7.3% of total time)
5. controller step - create house (6.7% of total time)
6. controller step - set object filter (4.4% of total time)

house_generator - add_floor_objects (4.4% of total generation time)

Here's a speedscope profiling log for the above run.

Can someone please help with optimizations of these ai2thor methods?

How to eval the procthor-model in https://github.com/allenai/procthor-models?

I run the

python allenact/main.py projects/objectnav_baselines/experiments/robothor/clip/objectnav_robothor_rgb_clipresnet50gru_ddppo.py -o storage/objectnav_10k_rgb_resnet50gru_ddppo -c pretrained_model_ckpts/procthor-models/scale_ablation/exp_10k-houses__stage_02__steps_000195225491.pt --eval

And i got this error

    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for ResnetTensorNavActorCritic:
        Unexpected key(s) in state_dict: "prev_action_embedder.fc.weight". 
        size mismatch for goal_visual_encoder.embed_goal.weight: copying a param with shape torch.Size([16, 32]) from checkpoint, the shape in current model is torch.Size([12, 32]).
        size mismatch for state_encoders.single_belief.rnn.weight_ih_l0: copying a param with shape torch.Size([1536, 1574]) from checkpoint, the shape in current model is torch.Size([1536, 1568]).

It looks like the configuration file you are using for training on procthor-10k is different from objectnav_robothor_rgb_clipresnet50gru_ddppo.py. How should I modify the file?

visualization

Hi ,

I hope this message finds you well. I'm writing to ask about the temp.json generated,I want to visualize the data in unity.Thank you in advance for your assistance.

Displaying Floor Objects in House Plans: Excluding Surfaces and Agents

Dear author,

Your work is excellent. I would like to know how to display a bird's-eye view of a house that only includes floor objects, without showing surface objects and agents.

Thanks!
Lu

ROS Support or ROS Bridge

Do you have any support for ROS/ROS2 or ROS bridge?

'Unexpected path error' after upgrading house

Hi,

I upgraded the houses in the train subset of the house scenes loaded from 'procthor-10k'. I used the upgrade_house_version.py script for the same.

I however get the following error. Can someone help me understand how can I resolve the same? Is this a bug?

File "main_real.py", line 223, in <module>
    main()
  File "main_real.py", line 179, in main
    currect_house = SCENES.initialize_house(
  File "/home/sohamdit/FILM_refactor_Alexa/./boolset/tasks_and_samplers.py", line 240, in initialize_house
    controller.step(
  File "/home/sohamdit/anaconda3/envs/py38/lib/python3.8/site-packages/ai2thor/controller.py", line 1011, in step
    raise RuntimeError(
RuntimeError: Newtonsoft.Json.JsonReaderException: Error reading string. Unexpected token: StartObject. Path 'house.proceduralParameters.ceilingMaterial'.
  at Newtonsoft.Json.JsonReader.ReadAsString () [0x0009c] in <f2d82e1ac3694e6399a979c083d80d79>:0 
  at Newtonsoft.Json.JsonReader.ReadForType (Newtonsoft.Json.Serialization.JsonContract contract, System.Boolean hasConverter) [0x000c2] in <f2d82e1ac3694e6399a979c083d80d79>:0 
  at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject (System.Object newObject, Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, System.String id) [0x00266] in <f2d82e1ac3694e6399a979c083d80d79>:0 
  at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x00161] in <f2d82e1ac3694e6399a979c083d80d79>:0 
  at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x0006d] in <f2d82e1ac3694e6399a979c083d80d79>:0 
  at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue (Newtonsoft.Json.Serialization.JsonProperty property, Newtonsoft.Json.JsonConverter propertyConverter, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty, Newtonsoft.Json.JsonReader reader, System.Object target) [0x00069] in <f2d82e1ac3694e6399a979c083d80d79>:0 
  at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject (System.Object newObject, Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, System.String id) [0x00266] in <f2d82e1ac3694e6399a979c083d80d79>:0 
  at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x00161] in <f2d82e1ac3694e6399a979c083d80d79>:0 
  at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x0006d] in <f2d82e1ac3694e6399a979c083d80d79>:0 
  at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize (Newtonsoft.Json.JsonReader reader, System.Type objectType, System.Boolean checkAdditionalContent) [0x000db] in <f2d82e1ac3694e6399a979c083d80d79>:0 
  at Newtonsoft.Json.JsonSerializer.DeserializeInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType) [0x00054] in <f2d82e1ac3694e6399a979c083d80d79>:0 
  at Newtonsoft.Json.JsonSerializer.Deserialize (Newtonsoft.Json.JsonReader reader, System.Type objectType) [0x00000] in <f2d82e1ac3694e6399a979c083d80d79>:0 
  at Newtonsoft.Json.Linq.JToken.ToObject (System.Type objectType, Newtonsoft.Json.JsonSerializer jsonSerializer) [0x00012] in <f2d82e1ac3694e6399a979c083d80d79>:0 
  at Newtonsoft.Json.Linq.JToken.ToObject (System.Type objectType) [0x002ff] in <f2d82e1ac3694e6399a979c083d80d79>:0 
  at ActionDispatcher.Dispatch (System.Object target, DynamicServerAction dynamicServerAction) [0x00180] in <9aa269828bcd47a997197f4911198591>:0 
  at UnityStandardAssets.Characters.FirstPerson.BaseFPSAgentController.ProcessControlCommand (DynamicServerAction controlCommand, System.Object target) [0x0007d] in <9aa269828bcd47a997197f4911198591>:0

Generating procthor-10k dataset from procthor-main branch

Hi,

I was facing a few issues while generating the same formats across procthor-10k and procthor. I'm currently using the main branch.

Comparing two jsons from procTHOR and procTHOR-10k, the objects["children"]["id"] does not have object types as in the ones generated from procTHOR.

JSoN from procthor main branch:

    "objects": [
        {
            "assetId": "Dresser_315_1",
            "children": [
                {
                    "assetId": "Vase_Medium_3",
                    "id": "small|4|2",

JSoN from procthor-10k branch:

    "objects": [
        {
            "assetId": "Countertop_I_8x2",
            "children": [
                {
                    "assetId": "Apple_24",
                    "id": "Apple|surface|2|0",

Which branch from procthor should I use to generate the dataset in the same format as procthor-10k?

Unity Backend code availability?

Is the Unity backend code also available for generating the Houses from the jsons?

More room types (e.g corridor)

Hello, currently with procthor, only 4 room types seem to be supported (bedroom, bathroom, kitchen and living room). Is there any way to build a scene with other room types (like a corridor to connect some rooms) to have a layout closer to reality?

Thanks

Matthieu

Small objects are not on the surface of receptacles

I loaded the json file generated by procthor in unity according to this method
https://github.com/allenai/ai2thor/issues/1128
but the position of some small objects seems to be wrong, they are not on the surface of receptacles.

The following image is from the 8th scene in the train folder of the procthor 10k dataset. Mugs, laptops and teddy bears are not spawning in the correct locations.

Documentation/Description

Hi team,
It was mentioned in the ProcTHOR paper that the dataset .json files would be released as open source, when can we expect it?
If not, is there a documentation of the toolset to create a dataset using ProcTHOR? I'd really like to know if there is one, as it's a bit confusing to deal with the raw code right now.

Where can I get full object list?

Hi!
As paper mentioned that ProcTHOR includes 1,633 interactive household objects across 108 categories, where can I get their name?
And when navigating in a scene, how can I access to the all object names used in the scene?

Customized Asset Placement in room generation: Selecting and Positioning specific items

Hello everyone, I have questions about placing assets during generation.
Is there a method to include only a specific list of assets in a room or apartment? For instance, generating a kitchen with only ['fridge', 'diningTable'].
If this is possible, can these assets be positioned next to each other?

error when "python scripts/example.py"

@Lucaweihs
Traceback (most recent call last):
File "scripts/example.py", line 6, in
house, _ = house_generator.sample()
File "/home/haha/桌面/procthor-main/procthor/generation/init.py", line 131, in sample
self.controller = Controller(quality="Low", **PROCTHOR_INITIALIZATION)
File "/home/haha/anaconda3/envs/procthor/lib/python3.8/site-packages/ai2thor/controller.py", line 513, in init
self._build = self.find_build(local_build, commit_id, branch, platform)
File "/home/haha/anaconda3/envs/procthor/lib/python3.8/site-packages/ai2thor/controller.py", line 1306, in find_build
raise ValueError(
ValueError: Invalid commit_id: f0825767cd50d69f666c7f282e54abfe58f1e917 - no build exists for arch=Linux platforms=Linux64
can you please tell me how to solve this?

[BUG] Teleport failed and no error message reported

Hi, I find that in the following case, the teleport operation failed to set the robot location and there is no error message.
Could you please help me figure out what is wrong?

Thanks!

Code:

from ai2thor.controller import Controller
from ai2thor.platform import CloudRendering

import prior

WIDTH, HEIGHT = 600, 600
FOV = 90
ROT_STEP = 60
GRID_SIZE = 1.5  # 1.5

dataset = prior.load_dataset("procthor-10k")

controller = Controller(scene=dataset['train'][0],
                        width=WIDTH,
                        height=HEIGHT,
                        fieldOfView=FOV,
                        gridSize=GRID_SIZE,
                        renderDepthImage=True,
                        renderInstanceSegmentation=True,
                        platform=CloudRendering)



positions = controller.step(action="GetReachablePositions").metadata["actionReturn"]
position = positions[1]
rotation = 30
event = controller.step(action="Teleport", position=position, rotation=rotation)
print("Teleporting the agent to", position, " with rotation", rotation)
print("Agent position: ", event.metadata['agent']['position'])
print("Agent rotation: ", event.metadata['agent']['rotation'])

Result:
We can see that the expected location and rotation of the agent are different from the actual location and rotation.

Teleporting the agent to {'x': 5.0, 'y': 0.9009997844696045, 'z': 2.0}  with rotation 30
Agent position:  {'x': 3.5, 'y': 0.9009997844696045, 'z': 2.0}
Agent rotation:  {'x': -0.0, 'y': 90.0, 'z': 0.0}

How to use the procthor-models

Dear author,
I found the model weights for ProcTHOR experiments in the project procthor-models, but I do not know how to use this model, such as the architecture of the model and the input and output of the model, can you provide more information about how to use them?
Thanks,
Juan

How to run Locobot ObjectNav in ProcThor?

We'd like to train and evaluate the simulated Locobot on object navigation in ProcThor homes as reported in the ProcThor paper. The infrastructure for this seems to be split into different GitHub repositories. Could you help us understand which pieces we need from which repository to replicate your experimental setup?

Given my current understanding, I think the procthor repository only generates homes but doesn't simulate them. So we could either generate homes using procthor or use the pregenerated procthor-10k dataset in prior. We would then somehow load those houses into ai2thor and combine them with the locobot from robothor-challenge, and for evaluation use the object-nav-eval dataset from prior?

Thanks a lot for your help!

Objects flying in the air

Hi @mattdeitke and @Lucaweihs!
I've tried to generate my own procthor dataset (1-room only houses) and got the environments, which have some flying objects. See example below:

Why could it happen and how can I avoid it?
Looking forward to your reply! Thank you!

Procthor documentation

Is there any documentation for procthor, like there is one for ai2thor?
https://ai2thor.allenai.org/ithor/documentation

making 3d scenes compatible with blender

Since unity3d is somewhat limited in its capability for rendering scenes, do you think it would be possible to export the scenes as fbx, usd, or glb scenes? If this is already possible, can someone point me towards the solution?

Realistic thicker walls

Thanks again for this amazing work!
I was wondering if it is possible to give the walls a certain thickness because right now they are only one point thick.
Alternatively, you could have a small gap between two sides of the planar walls. Is there a parameter for this?

Method for saving all images in the scene

Dear author,
your work is excellent. I want to know if is there any method to save all images in the house at once. I try to access the image by teleporting the agent to the all reachable position, but this method is too slow.
Thanks
Juan

room_type_map error

Hi!

I'm trying to generate the dataset which will contain only 1-room houses. For that I have edited room_specs.py file by saving only 4 room specs available:

        RoomSpec(
            room_spec_id="kitchen",
            sampling_weight=1,
            spec=[LeafRoom(room_id=2, ratio=1, room_type="Kitchen")],
        ),
        RoomSpec(
            room_spec_id="living-room",
            sampling_weight=1,
            spec=[LeafRoom(room_id=2, ratio=1, room_type="LivingRoom")],
        ),
        RoomSpec(
            room_spec_id="bedroom",
            sampling_weight=1,
            spec=[LeafRoom(room_id=2, ratio=1, room_type="Bedroom")],
        ),
        RoomSpec(
            # scale=1.25?
            room_spec_id="bathroom",
            sampling_weight=1,
            spec=[LeafRoom(room_id=2, ratio=1, room_type="Bathroom")],
        ),

During the generation process after the random number of generated houses I'm encountering an error like:

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/mkhan/.conda/envs/pr/lib/python3.9/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/mkhan/.conda/envs/pr/lib/python3.9/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/home/mkhan/pr/scripts/generate_procthor_10k_dataset.py", line 63, in generate_house
    house, _ = house_generator.sample()
  File "/home/mkhan/pr/procthor/generation/__init__.py", line 201, in sample
    door_polygons = gfs.add_doors(
  File "/home/mkhan/pr/procthor/generation/doors.py", line 103, in default_add_doors
    outdoor_openings = select_outdoor_openings(
  File "/home/mkhan/pr/procthor/generation/doors.py", line 139, in select_outdoor_openings
    room_type = room_type_map[room_id]
KeyError: 0
The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/mkhan/pr/scripts/generate_procthor_10k_dataset.py", line 81, in <module>
    r = p.map(generate_house, range(100_000))
  File "/home/mkhan/.conda/envs/pr/lib/python3.9/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/home/mkhan/.conda/envs/pr/lib/python3.9/multiprocessing/pool.py", line 771, in get
    raise self._value
KeyError: 0
"""

@mattdeitke Can you please help to understand what can cause it or what's wrong with my set up?

Bug In action="GetReachablePositions" For ProcTHOR Scenes

Hello again ProcTHOR Team,

Thanks for your quick responses and helpful tips for my previous questions about your dataset! I am encountering a different issue, relating to getting all reachable positions in a ProcTHOR scene once instantiated. Here is a snippet for reproducibility:

from ai2thor.controller import Controller
from ai2thor.platform import CloudRendering
import prior


if __name__ == '__main__':

    data = prior.load_dataset(
        "procthor-10k", revision="rearrangement-2022"
    )["train"]

    controller = Controller(
        commit_id="90eac925dc750818890069e3131f899998dc58b4",
        platform=CloudRendering,
    )

    metadata = controller.reset(scene=data[0], renderInstanceSegmentation=True)
    print(metadata)

    metadata = controller.step(action="GetReachablePositions")
    print(metadata)

The results of the two print statements:

<ai2thor.server.Event at 0x7f9c0ee9de50
    .metadata["lastAction"] = CreateHouse
    .metadata["lastActionSuccess"] = True
    .metadata["errorMessage"] = "
    .metadata["actionReturn"] = None
>
<ai2thor.server.Event at 0x7f9c0ee9dfa0
    .metadata["lastAction"] = GetReachablePositions
    .metadata["lastActionSuccess"] = False
    .metadata["errorMessage"] = "InvalidOperationException: In Procedural, position (-0.8, -38.9, 0.5) can be reached via capsule cast but is beyond the scene bounds.. trace:   at UnityStandardAssets.Characters.FirstPerson.BaseFPSAgentController.getReachablePositions (System.Single gridMultiplier, System.Int32 maxStepCount, System.Boolean visualize, System.Nullable`1[T] gridColor, System.Single gridWidth, System.Boolean directionsRelativeAgent) [0x0030d] in <6afd8f78be764eeba7be30f178fa1cb8>:0    at UnityStandardAssets.Characters.FirstPerson.BaseFPSAgentController.GetReachablePositions (System.Nullable`1[T] maxStepCount, System.Boolean directionsRelativeAgent) [0x0002e] in <6afd8f78be764eeba7be30f178fa1cb8>:0    at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)   at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00032] in <695d1cc93cca45069c528c15c9fdd749>:0 
    .metadata["actionReturn"] = None
>

The first event (scene creation) works, but there seems to be a check within the GetReachablePositions action that is failing. Specifically, it sounds like "(-0.8, -38.9, 0.5) can be reached via capsule cast but is beyond the scene bounds" suggests that the action is attempting to emit a point that is out of the scene bounds, which is being checked internally by the simulator.

Other actions, like "MoveAhead" seem to work fine.

Thanks for helping debug,
Brandon

import error when run example.py

Hi I got one error when I run example.py:

ImportError: cannot import name 'Attribute' from 'attr'

I run pip install attr when it says cannot find attr, but I am not sure if I installed the correct package. Thanks.

House model export to Unity

Hi AI2 Team,
I want to put the generated house model into a virtual reality environment to interact with it. Is there a way to export the model to a Unity scene?
Thanks a lot.

GoDot instead of Unity backend

Is it possible to use GoDot as a backend instead of Unity?

Question about how to run example.py

when I run example.py, I get error as shown in the figure. Xlib.error.ConnectionClosedError: Display connection closed by server: [Errno 104] Connection reset by peer. How to solve this problem?

visible object can't found in event.instance_detections2D or frame

I search all objects in event.metadata["objects"]，to get the visible objects' bbox
But I found that many visible object can't found in event.instance_detections2D or frame
here's an example
Bed and CD are visible.
*visible means metadata[object]['visible'] = True

How to get a shortest path to an object?

Hi, I would like to ask how to get a shortest path to the object.
I use the code from https://ai2thor.allenai.org/robothor/documentation/#navigation :

from ai2thor.util.metrics import (
    get_shortest_path_to_object_type
)

path = get_shortest_path_to_object(
   controller=controller,
    object_type="Apple",
    initial_position=dict(
        x=0,
        y=0.9,
        z=0.25
    )
)

But the path point returned by this code is not on the grid point.
Thanks!!!

About the return value of event.instance_detections2D

I would like to know if the function event.instance_detections2D can return values that are not integers,such as float

"Unable to CreateHouse!" Error when running example.py

Unable to CreateHouse!
  File "/home/nichols/mambaforge/envs/ll4ma/lib/python3.8/site-packages/procthor/generation/small_objects.py", line 85, in add_small_objects
    assert event, "Unable to CreateHouse!"
  File "/home/nichols/mambaforge/envs/ll4ma/lib/python3.8/site-packages/procthor/generation/__init__.py", line 209, in sample
    add_small_objects(
  File "/home/nichols/code/procthor/scripts/example.py", line 6, in <module>
    house, _ = house_generator.sample()
  File "/home/nichols/mambaforge/envs/ll4ma/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/nichols/mambaforge/envs/ll4ma/lib/python3.8/runpy.py", line 194, in _run_module_as_main (Current frame)
    return _run_code(code, main_globals, None,
AssertionError: Unable to CreateHouse!

Followed the procthor readme.md. Getting this error when running. I'm hoping to export these asset arrangements to another simulation engine. Running in a conda environment on ubuntu 20.

How is the asset library tied to the code?

Can someone point me to the code where the json generation ties with the asset/material library?
I am looking to tie my own Unity Asset library with the json generation codebase. If at all that is possible?

Unable to CreateHouse example.py

Hi,

Running the example script gives the following error:

Traceback (most recent call last):
  File "/home/sirdome/EmbodiedProject/new/procthor/scripts/example.py", line 6, in <module>
    house, _ = house_generator.sample()
  File "/home/sirdome/EmbodiedProject/new/procthor/procthor/generation/__init__.py", line 287, in sample
    gfs.add_small_objects(
  File "/home/sirdome/EmbodiedProject/new/procthor/procthor/generation/small_objects.py", line 83, in default_add_small_objects
    assert event, "Unable to CreateHouse!"
AssertionError: Unable to CreateHouse!

The event error message gives:

<ai2thor.server.Event at 0x7f4fc89100a0
    .metadata["lastAction"] = CreateHouse
    .metadata["lastActionSuccess"] = False
    .metadata["errorMessage"] = "Newtonsoft.Json.JsonSerializationException: Error converting value "MediumWoodSmooth" to type 'Thor.Procedural.Data.MaterialProperties'. Path 'house.rooms[0].floorMaterial'. ---> System.ArgumentException: Could not cast or convert from System.String to Thor.Procedural.Data.MaterialProperties.   at Newtonsoft.Json.Utilities.ConvertUtils.EnsureTypeAssignable (System.Object value, System.Type initialType, System.Type targetType) [0x00067] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Utilities.ConvertUtils.ConvertOrCast (System.Object initialValue, System.Globalization.CultureInfo culture, System.Type targetType) [0x00036] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.EnsureType (Newtonsoft.Json.JsonReader reader, System.Object value, System.Globalization.CultureInfo culture, Newtonsoft.Json.Serialization.JsonContract contract, System.Type targetType) [0x00117] in <f2d82e1ac3694e6399a979c083d80d79>:0     --- End of inner exception stack trace ---   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.EnsureType (Newtonsoft.Json.JsonReader reader, System.Object value, System.Globalization.CultureInfo culture, Newtonsoft.Json.Serialization.JsonContract contract, System.Type targetType) [0x00149] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x000dc] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue (Newtonsoft.Json.Serialization.JsonProperty property, Newtonsoft.Json.JsonConverter propertyConverter, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty, Newtonsoft.Json.JsonReader reader, System.Object target) [0x00069] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject (System.Object newObject, Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, System.String id) [0x00266] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x00161] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x0006d] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateList (System.Collections.IList list, Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonArrayContract contract, Newtonsoft.Json.Serialization.JsonProperty containerProperty, System.String id) [0x00173] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, System.Object existingValue, System.String id) [0x001c4] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x0007f] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue (Newtonsoft.Json.Serialization.JsonProperty property, Newtonsoft.Json.JsonConverter propertyConverter, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty, Newtonsoft.Json.JsonReader reader, System.Object target) [0x00069] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject (System.Object newObject, Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, System.String id) [0x00266] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x00161] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x0006d] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize (Newtonsoft.Json.JsonReader reader, System.Type objectType, System.Boolean checkAdditionalContent) [0x000db] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.JsonSerializer.DeserializeInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType) [0x00054] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.JsonSerializer.Deserialize (Newtonsoft.Json.JsonReader reader, System.Type objectType) [0x00000] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Linq.JToken.ToObject (System.Type objectType, Newtonsoft.Json.JsonSerializer jsonSerializer) [0x00012] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Linq.JToken.ToObject (System.Type objectType) [0x002ff] in <f2d82e1ac3694e6399a979c083d80d79>:0    at ActionDispatcher.Dispatch (System.Object target, DynamicServerAction dynamicServerAction) [0x00180] in <688391d437854f3da453f64e1df76e9d>:0    at UnityStandardAssets.Characters.FirstPerson.BaseFPSAgentController.ProcessControlCommand (DynamicServerAction controlCommand, System.Object target) [0x0007d] in <688391d437854f3da453f64e1df76e9d>:0 
    .metadata["actionReturn"] = None

how do i fix this issue?

Some ProcTHOR Files May Be Corrupted

Hello AI2 Team,

This issue references the rearrangement extension of procthor (it looks like issues can't be filed on that, perhaps since its a template):

https://github.com/jordis-ai2/ai2thor-rearrangement-procthor/tree/procthor-2022

It seems like certain files in data/2022procthor are corrupted. See the below steps to reproduce:

git clone https://github.com/jordis-ai2/ai2thor-rearrangement-procthor.git -b procthor-2022
cd ai2thor-rearrangement-procthor/data/2022procthor/split_train
pip install compress_json

Then in a python interpreter:

>>> import compress_json
>>> compress_json.load("scene_names.json.gz")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/anaconda3/lib/python3.8/site-packages/compress_json/compress_json.py", line 195, in load
    json_content = json.load(file, **json_kwargs)
  File "/home/user/anaconda3/lib/python3.8/json/__init__.py", line 293, in load
    return loads(fp.read(),
  File "/home/user/anaconda3/lib/python3.8/gzip.py", line 292, in read
    return self._buffer.read(size)
  File "/home/user/anaconda3/lib/python3.8/gzip.py", line 479, in read
    if not self._read_gzip_header():
  File "/home/user/anaconda3/lib/python3.8/gzip.py", line 427, in _read_gzip_header
    raise BadGzipFile('Not a gzipped file (%r)' % magic)
gzip.BadGzipFile: Not a gzipped file (b've')

I get the same error when attempting to train your procthor model:

allenact -b baseline_configs/one_phase/procthor one_phase_rgb_clip_dagger \
 -s 12345 --config_kwargs '{"distributed_nodes":1}'

Thanks for helping resolve this!

Brandon

Door and window openness randomization

I saw that it seems not to be possible to randomize the door and/or window openness with procthor since the openness of doors only has two states (open, closed). Are there any plans to support continuous openness states of doors/windows at some point? Would that be even possible and if so, how much effort would it take to implement that?

RemoveFromScene method is not working

During the time of executing RemoveFromScene method in procthor, the execution of the environment is paused. Please tell how to solve this problem?

AssertionError: Unable to CreateHouse!

i am facing an issue tying to run the basic example.py scrip

`from procthor.generation import PROCTHOR10K_ROOM_SPEC_SAMPLER, HouseGenerator

house_generator = HouseGenerator(
split="train", seed=42, room_spec_sampler=PROCTHOR10K_ROOM_SPEC_SAMPLER
)
house, _ = house_generator.sample()
house.validate(house_generator.controller)

house.to_json("temp.json")
`

#error details
`---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
Cell In[1], line 6
1 from procthor.generation import PROCTHOR10K_ROOM_SPEC_SAMPLER, HouseGenerator
3 house_generator = HouseGenerator(
4 split="train", seed=42, room_spec_sampler=PROCTHOR10K_ROOM_SPEC_SAMPLER
5 )
----> 6 house, _ = house_generator.sample()
7 house.validate(house_generator.controller)
9 house.to_json("temp.json")

File ~/anaconda3/envs/ai2thor2/lib/python3.8/site-packages/procthor/generation/init.py:209, in HouseGenerator.sample(self, partial_house, return_partial_houses)
207 if partial_house.next_sampling_stage <= NextSamplingStage.SMALL_OBJS:
208 with advance_and_record_partial(partial_house):
--> 209 add_small_objects(
210 partial_house=partial_house,
211 rooms=partial_house.rooms,
212 split=self.split,
213 controller=self.controller,
214 )
215 small_objects = [
216 *partial_house.objects[len(floor_objects) + len(wall_objects) :]
217 ]
218 randomize_object_colors(objects=small_objects)
...
---> 85 assert event, "Unable to CreateHouse!"
86 controller.step(action="SetObjectFilter", objectIds=[])
88 # NOTE: Get the objects in the room.

AssertionError: Unable to CreateHouse!`

#System info :

22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Apr 4 14:39:20 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

allenai / procthor Goto Github PK

procthor's Issues

Description

Recommend Projects

Recommend Topics

Recommend Org