Comments (13)
Thanks Diego :)
I've never heard of this environment before, but it seems possible if someone wants to take the time to implement it. Can you explain more about the mechanics? Ie: when do cells become burning? What does it mean for a cell to be protected?
from minigrid.
Absolutely! :-) Let's consider a NxN
grid where (for the sake of simplicity) a single cell c_{1}
has status = BURNING
and all others c_{2}...c_{n}
have status = NOT_BURNING
. A firefighter agent FF
is randomly placed at a cell c_{x}
, which (consequently) has status = PROTECTED
. Thus, the simulation starts with 1 BURNING
cell, 1 PROTECTED
cell (where the agent is) and t-2
NOT_BURNING
cells, where t
=total cells.
In a minimalistic config. at each time step:
- the fire spreads (i.e., all neighbor cells of
c_{1}
catch fire -> status=BURNING
) - the
FF
agent protects one cell (status =PROTECTED
) - the number of
NON_BURNING
cells is updated.
- agent's actions are standard = up, down, left, right
The objective of the game is to enclose and (consequently) stop the fire.
There are some derivations of this game (e.g., to prevent the fire to touch a highway), but this would be (one of) the simplest cenarios.
from minigrid.
I think the easiest way to implement this sort of scenario without adding a new kind of object type would be to use colored floor tiles, and have each color mean something different, eg: red means burning, blue means not burning. Note that in MiniGrid the agent normally has a partially observable view and it can turn left/right.
Would every cell on the grid be able to catch fire, or only some cells which can burn?
The agent can protect a cell and stop it from burning by standing on it, but can this cell burn again once the agent moves away?
Do you have a link to another implementation, or pictures?
from minigrid.
Normally every cell which is not protected can burn. And as soon as the agent protects the cell, it can not change its status (i.e., protected). So this would be the minimalistic version of the game. Yes, so I have a running implementation here, but was considering to explore your framework. Initial results using Q-Learning were not promising though.
from minigrid.
Seems like if all neighboring cells to a burning cell catch fire, and the agent can only move one cell at a time, everything will be burning pretty fast, no? Like, if you have an 8x8 grid, and you set the corner on fire, then in just 7 steps everything is burning?
I can't see your implementation, seems it's private.
from minigrid.
so, there is a concept of budget
here too (I didn't mention before to keep it simple). But one can define a value for this variable (e.g., 1.9) so that, at each time step, you can protect budget + residue
cells. For instance, t=1 the FF agent can protect math.floor(1.9+0.0)
= 1 cell (0.9 left). t=2 the FF agent can protect math.floor(1.9+0.9)
= 2 cells (0.8 left)...
But overall, in the highway game (FF agent must protect the highway) we can consider that the fire always starts at a position below (Y-axis) the FF agent and the highway is always located at the top position in the grid.
from minigrid.
Oh, sorrry, didn't notice that. I just made it public now should work :-)
from minigrid.
If you can protect multiple cells at each step, this environment seems not super weill suited for a gridworld, to be honest. It's more of a strategy game than something with an embodied agent.
from minigrid.
Hm...see, actually the protection action just changes the status of a cell, in practice (this was also the way I implemented before in OpenAI). But I could consider, for now, that the agent just protects 1 cell at each step. The problem is that unlike the examples I've found here, the environment, in this case, is dynamic. For instance, imagine the lava crossing game you have, but the lava moves at each iteration (like a fire would [let's not consider speed and other factors here :-)]) and you have to figure out a way to reach a certain cell X.
from minigrid.
To implement protection with multiple cells, I guess you could use the done action. The agent moves around, it uses the toggle action to protect the cell in front of it (or under it), and then it executes the done action when it has completed its "turn". Then the state of the world gets updated, and the agent gets to play again, protect more cells.
from minigrid.
OK, I will have a deeper look at the features of your library. Thanks! How would you model the fire (since it needs to spread)? As a single agent?
from minigrid.
You'd have to implement the logic in your environment class itself. I would write a function update_fire
and call it when self.actions.done
is executed by the agent. You'd override the step
function to intercept the done
and toggle
actions. There are examples of environments which override step
under gym_minigrid/envs/
.
The fire I would model as just colored Floor tiles if you want the agent to be able to walk over them. Red for burning, blue for not burning and green for protected or some such scheme.
from minigrid.
I'm going to close this for now because, after further consideration, it seems to me like this environment is too different from the other MiniGrid environments. It doesn't share the same actions and structure. I think it should be its own package. Feel free to fork MiniGrid if you find the code to be a useful starting point.
from minigrid.
Related Issues (20)
- [Question] Use of 'direction' in observation space? HOT 1
- [Bug Report] Toggling the box will make it disappear HOT 2
- [Bug Report] Error when creating BabyAI environment HOT 2
- [Question] Field of view algorithm explanation
- [Question] What do you mean by "done" action in the action space? HOT 3
- [Bug Report] Spec.to_json() of SimpleCrossing fails
- [Question] Getting total number of unique states HOT 1
- [Question] How to create vectorized envs for minigrid HOT 3
- [Bug Report] Unlock Pickup env is actually just Unlock HOT 4
- [Question] Difference between Minigrid and BabyAI envs HOT 1
- [Question] Is `ActionBonus` wrapper correct ? HOT 1
- [Question] How to build custom environment with custom action space HOT 1
- Python 3.11 support is missing from the README
- [Proposal] Update documentation HOT 3
- [Proposal] Add relevant project
- [Proposal] Discrete observations
- [Bug Report] ViewSizeWrapper ignored with RGBImgPartialObsWrapper HOT 1
- [Question] Get demonstration for 'MiniGrid-MultiRoom-N4-S5-v0' HOT 1
- [Bug Report] Cannot decode obs['image'] because of no implementation of agent in minigrid/core/world_object.py
- Accessing Specific Mission in BabyAI HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from minigrid.