It feels like benchmarks are missing the point about sim-ecs HOT 6 CLOSED

Hi @mayakwd ,

Thank you for taking interest in sim-ecs! The benchmarks were based on the Rust ECS Bench Suite. While I'm not happy with their workings and the tuning, yet, they seem to reflect changes in my code rather well and have helped me understand performance regressions in the past. Pitching them against other engines is a nice bonus, without too much benefit, though. The benchmark is synthetic and does not reflect how each library may perform in a real project.

The benchmarks are open-source and you can take a peek at how the code works. If you aren't happy with tick-knocks numbers, you can check the tick-knock bench code and look for any obvious mistakes or improvement potentials.

Just as well you can take a look at the sim-ecs simple iter bench code in order to understand how it is executed.

Unfortunately, I cannot comment on your test without actually seeing your code. Please provide it (ideally as an executable project) so I can take a look :)

from sim-ecs.

Thank you for the reply! I was looking for ways to optimize tick-knock. And was wondering why it is so slow compared to your library.
In the end, I found out that the numbers were just misleading, so I was chasing for nothing :)
So I've prepared a sample for you. It's just the same bench but with the counters and commented-out all benchmarks other than simple iteration.
Please check the code: https://github.com/mayakwd/sim-ecs/tree/benchmark-iterations-displaying
Or maybe I still don't understand the whole picture.

from sim-ecs.

I found the error. The reset code for the sim-ecs iter benchmark is wrong, which leads to the result described by you. I'll fix it in a future update. Thank you for testing and reporting the issue! I'll leave this open until I push the fix. Do you have any more issues or questions for this topic?

from sim-ecs.

No, thank you!

from sim-ecs.

@mayakwd I pushed the changes to my benchmarks and re-tested the iteration counts. In addition, I added some more tweaks to also re-enable the Scheduling test, which is not a simple iteration over entities, but also adds scheduling. It's a bit closer to real scenarios. I guess the next step would be to compare timings of the Pong example for a few frames with the different libraries, but to be honest, that sounds like a lot of work.

Anyway, the new results are in the README. While tick-knock looks good, it seems it has a deficite in scheduling, or I'm doing something wrong. Maybe you can check that and if my code is ok, I hope you can find performance-improvements in the code!

from sim-ecs.

Oh, thank you for the update! I'll take a look.

from sim-ecs.