katharostech / amethyst-ecs-benchmarks Goto Github PK

Benchmarks need to consider how they are allocating entities, as this can potentially impact both allocation and iteration performance.

i.e. Compare performance when allocating entities with the same component layout sequentially vs interleaved, and with or without batched allocation APIs if available.

For example, with three layouts: A(C1, C2), B(C1, C3) and C(C2, C3), compare performance of running 3 systems which each query one of the three components, with both batched and interleaved allocations of layouts A, B and C.

Legion does some work up-front to batch entity data by layout, so it is possible that specs is faster at allocations when not using the batching APIs, but legion is faster when using batched APIs (allocating a set of entities with the same layouts in one call). It would be interesting to benchmark the trade off here, as it is not always possible to perform all of your allocations ordered up-front.

Similarly, legion does this to ensure that iteration performance is stable regardless of how entities are allocated. In my experience, specs performance degrades quite significantly the more fragmented storage becomes as a result of allocating entities with different component types interleaved with each other.

One of the goals of this benchmark will be to make a real game as a benchmark. Essentially a stripped down version of Evoli, without much of any game logic.

An easier starting point might be something like @MaikKlein’s https://github.com/MaikKlein/bench_defense, made for pyro. But instead of 2D assets we use 3D (to cover 3D culling testing), and time control UI can be extracted out from Evoli.

I guess you’re waiting for OpenGL support to land before you can work with Amethyst >0.10?