Try another ECS implementation - ecs_pattern library

I wrote my own implementation of the ECS pattern:
https://github.com/ikvk/ecs_pattern
https://pypi.org/project/ecs-pattern/
And I used it to create an example: the Pong game.
The game uses pygame and my ecs_pattern library.
https://github.com/ikvk/ecs_pattern/tree/master/examples/pong

Try ecs_pattern in action!

===

Dear author of this lib, I hope for your understanding. No offense ;-)

I'm just getting started with esper and ECS systems in general.

I'm working on creating a simple multi agent simulation where various bots have various detectors and behaviors. In my simulation I would like to have the same detector component attached to an entity multiple times with each particular detector looking in a different direction.

As far as I can tell for ent, detector in self.world.get_component(DetectorComponent) only returns one detector for each entity, even if that entity has multiple detectors. How should I iterate over all of the detectors attached to an entity? How can I tell if a particular entity even has multiple detectors?

It would be great if you could upload a wheel version of this package to PyPI, which will make it possible to use esper with packaging systems that only support wheels and not source distributions. It should be as simple as:

python setup.py bdist_wheel
twine upload dist/esper-*.whl

Read the following code:

>>> import esper
>>> class Entity: pass
>>> class Animal(Entity): pass
>>> world = esper.World()
>>> world.create_entity(Entity())
1
>>> world.create_entity(Animal())
2
>>> world.get_component(Entity)
# a list has 1 item
>>> world.get_component(Animal)
# a list has 1 item

I want to use esper as the entities manager in a game. After I tried the above code, I think get_component should be:

>>> world.get_component(Entity)
# a list has 2 objects include an Entity and an Animal
>>> world.get_component(Animal)
# a list has 1 Animal object

which can return the inherited classes from given component.

I think the ECS may store not only the animal's position and its velocity but the whole animal, which is easy for designing the game.

The current release (0.9.8) is updated to Pypi, but the README was not updated to match.

After some time for real world testing, a new release should be pushed out. This will likely be v1.0.

How would i use the depth in a renderable component?

This file is needed for PEP 561 to mark the package as type-hinted. Without it the package isn't considered typed when installed as a package or from PyPI.

Hello,

Would it be possible to add a way to select entities based on components that they must not possess? If I want to select, for example, all entities except the player, I have to select them all, then make a specific case to check that the entity I'm looking at doesn't have a "player" component. It feels like an unnecessary step, since the get_components function is already doing the job of selecting entities.

It would be nice to have something along the lines of
World.get_components(self, *component_types: _Type[_Any], denied_component_types: tuple = None)
to allow such a filter at query time.

I do not know if this poses any design or performance issue, however, so I hope this issue isn't completely absurd.

It doesn't look like v1.5 is showing on pypi. Trying to install via pip results in 1.3 being installed by default. This means that try_component is not in alignment with the documentation and returns a generator, resulting in an error.

Hi everyone,

First and foremost, thank you for this elegant and beautifully crafted piece of code. We are in the process of developing a turn-based game on Discord and are considering using Esper for our project.

I have a question regarding the persistence and loading of the game world. Are there examples or recommendations on how to accomplish this? Should we focus solely on entities and components and then reconstruct the world context from them?

Thanks in advance for your assistance!

Hello,

Do you have any best practices to create relationships between entities ?

For instance I have a Farm component and a Plant Component

• I can create multiple entites with a Farm
• Create multiples entities with Plant
• I would like to link this Plants with a specific Farm

Not sure what would be the best things to do...

A list of Plants or Entity Ids on the Farm class ?

Thanks a lot !

Describe the bug
mypy show an error message in process methods because get_components gives "object" type.

To Reproduce

from dataclasses import dataclass
from esper import World, Processor


@dataclass
class Position:
    x: int = 0


@dataclass
class Velocity:
    x: int


class MovementProcessor(Processor):
    def process(self) -> None:
        for ent, (pos, vel) in self.world.get_components(Position, Velocity):
            pos.x += vel.x


world = World()
world.add_processor(MovementProcessor())
ent = world.create_entity(Position(), Velocity(1))
world.process()
pos = world.try_component(ent, Position)
if pos:
    print(f"player is now {pos.x}")

$ python esper_mypy_test.py 
player is now 1
$ mypy esper_mypy_test.py 
esper_mypy_test.py:18: error: "object" has no attribute "x"  [attr-defined]
Found 1 error in 1 file (checked 1 source file)

Expected behavior
mypy should find the components types from get_components

Development environment:

• Python 3.11.2
• esper==2.4.1
• mypy==1.1.1

Additional context
possibly related to #60

Hi Ben,

really pleased to use this simple python module, it’s concise and do the job.
It’s my first time trying ecs, so I wanted a quick suggestion on how to proceed, basically I’m trying to recreate for testing purpose Marvel Snap card game.
At the moment I understand exactly what entity and component are, little confused on how I should use systems in game like this, does every card needs its own system? Since each card have unique ability mostly.
Thanks in advance.

Daniele

Describe the bug
I'm trying to get started with Esper and am bumping into errors around the no longer existing World method.

    self.world = esper.World()
AttributeError: module 'esper' has no attribute 'World'

What's the new recommended way of doing this?

To Reproduce
Use esper.World()

Expected behavior
The documentation to reflect the code and what's required

Development environment:

• Python version 3.11
• Esper version 3

Within the next week or two, I was hoping to accomplish three things.

1. Spellcheck/Rewrite portions of the Readme, docstrings, comments etc.
2. Merge CachedWorld functionallity into World and delete CachedWorld
3. Hammer out a few proof of concepts for an event system.

Not exactly the typical issue, but I figured it would be the best way of communicating with you. I wanted to make sure these changes were welcome before I began.

It'd be nice if the verb as consistent.

Hey ! Thanks for this greet library !
In the README.rst I can read:

An event handler can be a function or class method.

That works fine with methods but with functions I get this error:

Python 3.10.4 (main, Mar 25 2022, 19:56:11) [GCC 10.2.1 20201203] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import esper
>>> def foo():
...     pass
... 
>>> esper.set_handler("foo", foo)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "~/.local/lib/python3.10/site-packages/esper/__init__.py", line 79, in set_handl
er
    event_registry[name].add(_WeakMethod(func, _make_callback(name)))
  File "/usr/lib/python3.10/weakref.py", line 52, in __new__
    raise TypeError("argument should be a bound method, not {}"
TypeError: argument should be a bound method, not <class 'function'>
>>> 

Hey @benmoran56 , first of all thanks for this nice library. I'm really enjoying using it!

I noticed that the type annotations for get_component and get_components are not quite correct.
Let's talk about get_component here, as it's easier. (get_components has the same plus another issue. I'll create a sep. issue for that one.)

get_component() is decorated with lru_cache, but the lru_cache swallows the type annotations (ref python/mypy#5107). As a result the type of the returned component cannot be inferred.

I type checked it with pyright, but other type checkers should yield similar results.

Potential Fix

A solution is to add type hits at type checking time (as suggested here python/mypy#5107):

from typing import TYPE_CHECKING
from typing import TypeVar as _TypeVar
if TYPE_CHECKING:
    # workaround to prevent the lru_cache to swallow the type annotations
    F = _TypeVar("F")
    def _lru_cache(f: F) -> F:
        pass
else:
    from functools import lru_cache as _lru_cache

Code to reproduce

Here is the code to reproduce the issue:

class A:
    def a(self):
        print("From A.a()")


class B:
    def b(self):
        print("From B.b()")


world = World()
world.create_entity(A(), B())


# With lru_cache on get_component -> type issues
for i, a in world.get_component(A):
    print(a.a())
    # Cannot access member "a" for type "object*"
    # Member "a" is unknown (reportGeneralTypeIssues)

# Without the lru cache on get_component -> no type issues
for i, a in world.get_component(A):
    print(a.a())

This piece of the README:

class Position(x=0, y=0)
    self.x = x
    self.y = y

is not valid Python -- it should have an __init__ function.

I am curious the reasoning behind why worlds were rolled up into the esper module? In working with this library and encountering this breaking change, I feel like I have to think a lot harder about this hidden context where as before it seemed much more explicit? Perhaps I am not thinking of world appropriately, which is why I feel the need to ask. Can you help me understand the reason behind the change? Perhaps it's my approach that hasn't been keeping in tune with the philosophy and design intent.

The benchmark should be enhanced to measure Entity creation/deletion performance.

In addition, benchmarking should be done of the lru_caching. For this, a specialized benchmark with a functional 60fps main loop might be best way. Possibly a seperate benchmark. A test might be:

1. A set main loop with X-number of queries per frame.
2. Measure the effect on query speed from adding or deleting 1 entity per 60 frames, 50 frames, 40 frames, etc. down to adding/deleting several entities per frame.

In the above test, it should show where the "tipping point" is for lru_cache usefulness with regards to the amount of Entity churn per frame.

Hello, I just wanted to ask: What are your thoughts on adding a Resource type to esper? Before esper I've used amethyst as an ECS for Rust. That system uses so called Resource to share data that is not specific to an direct entity. For example a global score that gets accessed by multiple systems.

The formal definition of a Resource by amethyst:

A resource is any type that stores data that you might need for your game AND that is not specific to an entity. For example, the score of a pong game is global to the whole game and isn't owned by any of the entities (paddle, ball and even the ui score text).

I know that I could just distribute the value to the systems at the creation time or simply add a new member to the world class. However, I believe it might be beneficial to have a shared resource marked as one by accessing them through world.get_resource(<Type>) for example. Do you have any thoughts on this?

Is there any plan to implement something equivalent to the Observer pattern (see for instance in entitas? They discuss about it in their Unite talk (from their readme), around the 19 minutes mark.

Essentially, it allows to register a Processor to react to a change in a given group (a set of entity with a given assembly of components). It can be either when a new entity enters, or leaves. This reduces often the amount of code needed to connect systems between them, and can also improves performance (no query is done by such a method if no changes have been detected).

To give an example, considered a system charged to generate a bunch of data the first time a terrain is created, and another system in charge of finding which terrains to load. One could link the two with an additional component, NewTerrainComponent, and have the second system add this tag, then the first system treat this terrain and remove the component.

Merging both into one system goes away, in my sense, from the single responsibility pattern of ECS. An observer system, however, could simply declare an action on a new entity being added to the group of entities with TerrainComponents.

It may be a somehow controversial feature - I think not everyone agree that it is part of an ECS system. It may be also necessary to introduce the concept of groups for this to work, so I realise it will not be a small one-time hack. This issue is just to know whether or not something like this would be considered at some point, or if esper is considered more or less feature complete as-is.

Describe the bug
As a factory method I assume it creates entity for sure but that do not happen is edge case of missing components.

To Reproduce

import esper

world = esper.World()
entity = world.create_entity()
assert world.entity_exists(entity)  # raises AssertionError

import esper

world = esper.World()
entity = world.create_entity()
assert not world.entity_exists(entity)  # not existing entity

entity_2 = world.create_entity()
assert entity_2 == 2  # Incrementing counter even though 1st one seems not to exist

Expected behavior
Imo proper behaviour is to be able to create "empty" entity and provide it with components later. Kind of how you do it with builder pattern.
Plus I've found that add_component() has logic for creating missing entity and that is most probably result of that bug. Following Single Responsibility Principle, that should not be part of add_component() method imo. See example below:

import esper

world = esper.World()
made_up_entity = 123
world.add_component(made_up_entity, None)  # Imo should raise KeyError because of missing entity
world.entity_exists(made_up_entity)  # Works just fine and should not

Development environment:

• Python 3.11
• Esper 2.4

Fix proposal
To fix that it is simple enough to pull https://github.com/benmoran56/esper/blob/master/esper/__init__.py#L227

if entity not in self._entities:
    self._entities[entity] = {}

out of for loop in create_entity() method.
Consider removing entity creation from add_component() method.

Tell me what do you think about it, I can implement change and unit tests later on

Describe the addition or change you'd like to see.
We should be running mypy or a similar type checker as part of the unit tests.

@SpotlightKid I am aware of the python2 branch at https://github.com/SpotlightKid/esper, but is it planned at all to open a pull-request for this? As I am using a library compatible only with Py2, it would be nice if I could simply add esper as a normal dependency, and not rely on a fork.

If the branch would use six, we could have a codebase supporting both, with simply the initial cost of setting it up, but no more worries after this.

Hello,

I was wondering why the world instance reference under the Processor class is not typed, i.e.,

`class Processor:
"""Base class for all Processors to inherit from.

Processor instances must contain a `process` method. Other than that,
you are free to add any additional methods that are necessary. The process
method will be called by each call to :py:class:`World.process`, so you will
generally want to iterate over entities with one (or more) calls to the
appropriate world methods there, such as::

    for ent, (rend, vel) in self.world.get_components(Renderable, Velocity):
         your_code_here()
"""

priority = 0
world = _Any

def process(self, *args, **kwargs):
    raise NotImplementedError`

with this set up I cannot get smart autocompletion on the world reference when working on a processor. (I'm using pycharm 2022.2.3)

I use my own Processor subclass, in which I've added the type information as

world: World = None

and the completion is working fine.

I haven't found any issue so far with this approach. Is there something I'm not considering? Why was the idea behind assigning it to the generic type Any?

Regards,
Alvaro

Readme: Readme needs rewritting and rewording. Multi-stage process and other parts of this issue are completed.
Docstrings: Docstrings need to be reworded, rewritten, reformated, and in some cases, added.
Might also add type hinting to certain methods.
Documentaton: Docs will be based mostly off of docstrings, with light edits dependin on context.

I'll add these as they come to mind. Some examples could be a version of processing the world that takes an argument, and only runs processes that take said argument.

This is mostly a shower thought, but two reasons a well-crafted short string id set would be better:

1. string-based dicts have their own hyper-optimised implementation in CPython (the implementation switches to a "general" dict as soon as you use a non-string key in the dict).
2. short strings constructed of random letters would be shorter than random numbers when displayed in debugging, and potentially could even use very short words / word combinations which could make debugging (displaying and recognising ids in a game UI for example) simpler.

The today esper API is class based programming. I think they are a unreasonable use of classes in the case of a python lightweight library. This API looks like this (from README.md).

import esper

class Position:
    def __init__(self, x=0.0, y=0.0):
        self.x = x
        self.y = y

class Velocity:
    def __init__(self, x=0.0, y=0.0):
        self.x = x
        self.y = y

class MovementProcessor(esper.Processor):
    def process(self):
        for ent, (vel, pos) in self.world.get_components(Velocity, Position):
            pos.x += vel.x
            pos.y += vel.y

world = esper.World()
world.add_processor(MovementProcessor())
player = world.create_entity(Velocity(x=0.9, y=1.2), Position(x=5, y=5))
world.process()  # repeat

The new API I suggest to write looks like this

import esper

class Position:
    def __init__(self, x=0.0, y=0.0):
        self.x = x
        self.y = y

class Velocity:
    def __init__(self, x=0.0, y=0.0):
        self.x = x
        self.y = y

def process_movement():
    for ent, (vel, pos) in esper.get_components(Velocity, Position):
        pos.x += vel.x
        pos.y += vel.y

esper.init_world()
esper.add_processor(process_movement)
player = esper.create_entity(Velocity(x=0.9, y=1.2), Position(x=5, y=5))
esper.process()  # repeat

Here is how I analyze the benefits and loss.

Readability

In process implementation, they are no unnecessary class X lines; they are no repeated self.world.
The world instance is module-side, they are no world attribute in the App-level object or in global variables.

Performance

esper implementation become straightforward.
Less calls to the dot operator in process implementation and even less using from esper import get_components.
contextvar.get at the start of every esper function must have a performance cost.
Need to write a benchmark to avoid taking a step back (first PR).

Modularity

It is harder to redefine World and Processor methods before use them to alter their behaviors. Because esper is a lightweight library, a developer with specific needs can implement a new function for this, so it is not so embarrassing.

Reusability

Using contextvars from the python standard library allows to use several World instances at the same time.

from contextvars import copy_context
import esper

def create_and_get():
    esper.create_entity(Position(0, 0))
    return list(esper.get_component(Position)


esper.init_world()
assert len(create_and_get()) == 1
cxt = copy_context()
cxt.run(exper.init_world)
assert len(cxt.run(create_and_get)) == 1
assert len(cxt.run(create_and_get)) == 2
assert len(create_and_get()) == 2

I used esper with pyglet and pymunk. So i create vertex_list for graphics component and bodies for physics components. When i removed those components i used del method to release objects. But cause of python, sometimes it crash an app. So i made child class of esper.World which call a components callback on delete_entity method. There i clean component before it leave the World

Thanks for the nice library!

As mentioned earlier, get_components has some type annotation issues.

First,
the lru_cache swallows the type annotation, see #59.
Let's assume the decorator is fixed/removed.

Second,
The argument *component_types: _Type[_C] says that we accept multiple component_typse,
more specifically it says that they *all have the same type _Type[_C].
However, each component type is different.
Same holds for the return (here without the _Type, just plain _C).

I'm not a python typing expert, but I think we would have to define multiple functions for the arity of aguments/elements to return.
Something like this:

from typing import TypeVar
C1 = TypeVar("C1")
C2 = TypeVar("C2")
C3 = TypeVar("C3")


def get_components(
    self, ct1: type[C1], ct2: type[C2]
) -> list[tuple[int, tuple[C1, C2]]]:
    return []


def get_components(
    self, ct1: type[C1], ct2: type[C2], ct3: type[C3]
) -> list[tuple[int, tuple[C1, C2, C3]]]:
    return []

def get_component(
    ...

Well, of course python does not allow to overwrite functions like this :)
One could hack something together or duplicate a bunch of code, but this might slow down esper too much.
Maybe you or somebody has an idea how to cleanly handle this issue.

When I use an ECS, I often find myself needing to perform some logic in the System/Processor when when a component is added or removed. For example, if a new PhysicsComponent is added to an entity, I may want to have a PhysicsProcessor add a physics object to a physics simulation.

I am currently looking at hacking up a Processor subclass to track which entities/components were processed on previous invocations of process() and figure out which entities are new or missing. Would there be interest in having something like this in Esper itself?

This would allow to:

• Upload the package to PyPI
• Automate unit tests with tox
• Install the package in a vitualenv
• etc.

I would create a PR, but I don't know how you like to organize the package metadata and what exactly to put for author, email, name etc.

I compiled Esper locally using Cython and experienced a mild speedup.

Perhaps the setup.py could look for Cython and if installed, compile Esper locally.

When using esper, I like to have all my processors decoupled. I mean no processor needs to python import another_processor to make his job. To achieve this, I use an event system.

I have a python package containing Event classes that have no dependencies, not on Component neither on Processor. A processor can import any Event and can publish it thanks to a function of World. Here is a snippet of a processor that only publishes an event (Maybe I've should used message instead of event :D ):

from event.my_event import AnEvent
class MyProcessor(Processor):
    def __init__():
        super().__init__()

    def process():
         self.world.publish(AnEvent())

Another snippet where a processor receive the event :

from event.my_event import AnEvent
class MySecondProcessor(Processor):
    def __init__():
        super().__init__()

    def process():
         for an_event in self.world.receive(AnEvent):
             do_something()

It could be great to have this mechanism inside the ECS library :

• each processor should have a method to publish/receive events.
• It is the responsibility of the World to ensure the dispatch of events to the processors.

I think the library would benefit from adding type hints. Some things, such as the type of get_components() will be harder to express, but I believe that most functions can be annotated rather easily.

Type hints would greatly enhance the auto-complete functionality of Python IDEs (e.g. when you access the world member variable of a Processor, you would get suggestions for all member functions of the World class) and also assure the user that they are using the library correctly.

I might add the type annotations myself if you think it is worth doing.

This was held off on until the api stabilized a bit. It's probably a good time to finally write them.

get_components does not get entities in order (ascending order). How can I get it in order?

I created about 5000 entities, and the order is out of order after 3932. Trying again is the same result.

def process(self, *args, **kwargs):
    if self.entity == None:
        for ent, (foo, foo2) in self.world.get_components(Foo, Foo2):
            self.entity = ent
            break

    if self.entity != None:
        foo_func()
        self.world.delete_entity(self.entity)
        self.entity = None

result...
...(1~3929)
entity:3930
entity:3931
entity:3932
entity:4096
...

I'm wondering if adding a try_components would be usefull to anyone.

Usage :

for hp, mp in self.world.try_components(ent, HP, MP):
    hp.value += 10
    mp.value += 10

I have a naive implementation running (with self.world as an esper.World):

   def try_components(self, *args):

        entity = args[0]
        args = args[1:]

        components = []
        for arg in args:
            if not self.world.has_component(entity, arg):
                return []
            else:
                components.append(self.world.component_for_entity(entity, arg))

        return [(*components, )]

Any tough / Feedback ?

Hey,

With an ECS architecture, I always have a PositionComponent in my game. This PositionComponent is quite simple, 2 integers representing a (x,y) position.

When using this component, I every time do something like :

  for ent,pos in self.world.get_component(PositionComponent):
      if pos.x() == x and pos.y() == y: 
          do_something()

It is not really performant neither elegant code.

I'd like a way to access specifically entities with a PositionComponent with given x and y. More technically, I think it could be interesting to have components that can be indexed by a parameter. In my example, the parameter will be (x,y).
I guess the code would become something like :

  for ent,pos in self.world.get_indexed_component(PositionComponent, (x,y)):
      do_something()

I didn't think about the technical implementation, just laying the idea here :)

I'm considering using esper in a project that requires multi-threading. I'd like to know if the library is designed to be thread-safe. Specifically, are there any examples or documentation on how to properly use esper in a multi-threaded environment?

Thank you for your time and effort in maintaining this library!

Hello,

i use esper for my roguelike game. I save entities in a sqlite database where i also save the entity id to have a relation between entities and the components.

Saving is pretty easy. Loading not so much. Let me explain

my db looks like this (shortend for better explanation)
Entity Table

Position Table

as you can see it pretty easy to see how these tables are releated. to get the position of an entity i query the db for the position where the entity_id is the same as in the Entity table.

As i found no other solution this is how i do it now:

1. I select the entity (i.e 13) with all it components in my db
2. create an entity with world.create_entity() (i.e i get 55 as id)
3. add all the components from entity 13 to the new entity 55
4. update the old entity id 13 in the db with the new id 55 and all its components that relates to id 13

that is pretty dirty imo. it would be awsome to create a "known entity" so i don't have to update the db everytime the id is changing.
i.e:

entity = world.create_known_entity(13)
world.add_component(entity, position)
...

sorry for my english. i tried my best to explain my problem. if there are any questions i can try to explain it better

Describe the bug
As for Single Responsibility Principle remove component should do just that but it removes empty entity as well.
As I'm creating tests for my own project, I need to populate entity with some dummy component to prevent that and test cleaning another component logic.

To Reproduce

import esper

class Component:
    ...


world = esper.World()
entity = world.create_entity(Component())

assert world.entity_exists(entity=entity)

world.remove_component(entity=entity, component_type=Component)

assert world.entity_exists(entity=entity)  # AssertionError

Expected behavior
I assume that cleaning components should not clear entity. As removing and adding component should be independent from entity existence. Entity should be deleted by delete_entity() only and not under the hood so programmer has full control over it.

Development environment:

• Python 3.11
• Esper 2.4

Additional context
I will create unit tests for it and fix if approved for implementation

Going into the actual reading the docstrings inside of methods could be replaced by actual documentation