High memory usage about akka.persistence.sql HOT 16 OPEN

The application is hosted as a windows service (using TopShelf for window service hosting, running http.sys as a web server). We do not use IIS.
The dump files you have seen above is the service running in console mode, from visual studio.

from akka.persistence.sql.

@tudorFrizeru I've taken a look at your dump file and it appears that the growth may be in/around unmanaged memory which should not typically be an issue with Akka.Persistence.Sql.

If I had to guess, I'm betting this is related to a couple of bugs in 8.0 around Native thread local storage cleanup (dotnet/runtime#95722) that won't get released till the 2024/02/13 (about a week from now). Examples of other people running into similar issues on .NET8: dotnet/runtime#96581
, dotnet/runtime#95922

Will keep digging in repo code however.

from akka.persistence.sql.

@to11mtm thank you for your quick response! I downgraded to .net 6 again and the problem remains the same.

Here's a screenshot with the managed memory allocations.
I can upload a new memory dump file if required.

from akka.persistence.sql.

@tudorFrizeru If you could that may help, there are a few other possibilities and that may narrow things down.

from akka.persistence.sql.

@to11mtm I added 2 new dump files at the same link :

• net6 Instrumentation
• net6 Memory Usage

I left them running for a long time, and they both pertain to the .net6 build

from akka.persistence.sql.

🤔

Question, is this running in IIS or via console?

from akka.persistence.sql.

@to11mtm FYI, I managed to isolate the problem to the Akka.Persistence.Sql package : I reverted back to using the Akka.persistence.SqlServer, with .net 8, and I have no memory issues anymore.

I will be happy to test again once there's a fix available!
Thank you for your support and for your great work!

from akka.persistence.sql.

@to11mtm FYI, I managed to isolate the problem to the Akka.Persistence.Sql package : I reverted back to using the Akka.persistence.SqlServer, with .net 8, and I have no memory issues anymore.

I will be happy to test again once there's a fix available! Thank you for your support and for your great work!

Thank you for that feedback!

I do have a PR up that I believe will solve some issues.

Was there anything in your app doing an EventsByTag query with a lot of existing 'in-filter' events (i.e. Thousands to tens of thousands) present after the starting offset? One issue I found, there was a missing limit on EventsByTag query, so cases like what I mentioned would potentially cause a lot of pressure on the Large Object Heap (Which would explain, likely quite a bit!)

from akka.persistence.sql.

@to11mtm has worked on some memory optimization improvements here around closures: #347

from akka.persistence.sql.

Hello everyone,
I encountered a similar issue and managed to resolve it in my scenario.
The problem stemmed from loading Entity Framework 8's DbContext in the constructor.

To avoid this, refrain from utilizing the DbContext property directly within the class. Instead, opt to use it within methods.

The recommended approach is to instantiate a new DbContext within each method call. You can achieve this either by creating a new instance explicitly (new DbContext()), or by retrieving it from the services (services.GetService()).

This practice ensures that each method operates with a fresh DbContext, preventing potential conflicts and ensuring better management of resources.

from akka.persistence.sql.

The problem stemmed from loading Entity Framework 8's DbContext in the constructor.

What does that have to do with this project? We use Linq2Db, not EF.

from akka.persistence.sql.

The problem stemmed from loading Entity Framework 8's DbContext in the constructor.

What does that have to do with this project? We use Linq2Db, not EF.

I think it has nothing to do with the Entity Framework, but with Net8.0 and the GC. In my case, the dbContext cache was not released by the GC as long as the class was used. In net4.8 it worked as expected. I thought there might be a similar problem with Linq2Db.

from akka.persistence.sql.