Auto-scaling Ring Buffer - also downscaling automatical? about gnet HOT 13 CLOSED

Please excuse the unnecessary confusion and unnecessary work.

Don't be sorry for that, feel free to open an issue when you encounter any problem when using gnet😄, it is my duty to help gnet users solve problems.

Also glad to hear that GC works, but still I have a doubt that Golang GC takes that long time(30 minutes) to collect garbages cuz GC is triggered by a fixed and short time(2 minutes) if the last GC finishes over 2 minutes ago, so it is still odd. But on the other hand, it also could be a GC mutator assist which might cause a long time pause to GC, maybe that is it.

Anyway, I will keep an eye on the GC in gnet, thanks for your discovery on this.

from gnet.

That is odd, it has nothing to do with the ring-buffer size but its number based on your desc, gnet will recycle ring-buffers after connections are closed, so it ought to save some memories in my expectation.

Could you tell it more detailedly about the I have 40k open websocket sessions, if I now reduce them to 20k? It could be some bugs in gnet and I want to ensure it.

from gnet.

Besides, which version of gnet are you using?

from gnet.

Hi panjf2000,

first of all: Thank you for your response. I use version v1.0.0-rc.4. I close the connections and gnet also sees that since it triggers OnClosed.

I benchmark my service by docker container. So I have the service running on the host with gnet and then spawn several containers that connect to the service on the host. Every connection is registered via OnOpened and every disconnect via OnClosed. Then I stop half of the containers to get 20k connections.

from gnet.

Could you use pprof tool to find out which part that occupies the most of memory? In that way, we will have an overall view about the memory usage of your gnet server.

from gnet.

At startup:

Opening 20k connections that send every second a text

Close 10k connections so we still have 10k connections

from gnet.

It seems that ring-buffers weren't GCed after connections were closed but I can't tell why that was happening because ring-buffers would be put back to sync.Pool which would be GCed at the next GC time, I will go investigate it further to figure out the root cause.

from gnet.

Or could you add some logs to see if the releasing code have been executed?

from gnet.

Added breakpoint, gets triggered:

from gnet.

Then I have no idea why those ring-buffers from closed connections weren't GCed, that makes no sense, I would have to take some time to investigate it.

from gnet.

Hi @panjf2000,

I run the test for longer time and it looks like it's working.

Here's what I did:

1. started the server
2. 40k connections established
3. 82 MB memory consumption
4. All connections disconnected
5. 82 MB memory consumption
6. 5 min. later: 166 MB memory consumption with the ring buffer
7. 30 minutes later:
   

Looks like sometimes the Golang GC takes longer.

Please excuse the unnecessary confusion and unnecessary work.

from gnet.

Hi @patrick-othmer

Have just discovered Gnet myself and am starting some work with websockets. Is the work you've done publicly available (eg github) or is closed source? Would definitely like to see what you've done (saves me from re-inventing the wheel).

Thanks

Ken

from gnet.

Hi @kpfaulkner,

sorry, it's closed source. But the gnet examples may help you: https://github.com/gnet-io/gnet-examples

Regards,

Patrick

from gnet.