Performance test occasionally fails with `fatal error: concurrent map writes` about agones HOT 4 CLOSED

Check the logs:
https://console.cloud.google.com/cloud-build/builds;region=global/275a3e64-934b-4512-80a4-28ff3ad733c8;step=6?e=13803378&mods=logs_tg_prod&project=agones-images

Error: Getting Permission denied for all resources.

from agones.

The error is coming from https://github.com/googleforgames/agones/blob/main/test/load/allocation/runscenario/runscenario.go#L126

Looking at the code, I'm curious how it ever works, since we have a gofunc per client and they are all writing to the same shared map.

from agones.

Check the logs:
https://console.cloud.google.com/cloud-build/builds;region=global/275a3e64-934b-4512-80a4-28ff3ad733c8;step=6?e=13803378&mods=logs_tg_prod&project=agones-images

Error: Getting Permission denied for all resources.

Sorry about that, probably not public, here's a copy
log-275a3e64-934b-4512-80a4-28ff3ad733c8.txt

Looking at the code, I'm curious how it ever works, since we have a gofunc per client and they are all writing to the same shared map.

I guess it doesn't manage to synchronously mutate the map too often.

Some quick ideas, not sure which ones are best:

• Some mutexes in the right places
• There are a few sync Maps that are thread safe out there, could go pick one (one with generics support?)
• Move stuff to channels?

from agones.

I guess it doesn't manage to synchronously mutate the map too often.

Some quick ideas, not sure which ones are best:

• Some mutexes in the right places
• There are a few sync Maps that are thread safe out there, could go pick one (one with generics support?)
• Move stuff to channels?

I think placing mutex in the right places should work. I have created a small example for this use case: https://goplay.tools/snippet/N84X6jIZmIq

We can utilize the syncmap but syncmap is slow and has poor performance. Personally, I haven't done any benchmark testing but this is what i got from the internet.

from agones.