Git Product home page Git Product logo

Comments (7)

dougwilson avatar dougwilson commented on August 15, 2024 1

It doesn't matter how big or small it is: that is why I will take a memory dump, because you'll be able to see a leak of any size easily since the number of remaining objects will be the same regardless of their size 👍

from timeout.

dougwilson avatar dougwilson commented on August 15, 2024

Due to how the gc works there shouldn't be a memory leak. Can you provide an app that demonstrates a memory leak so I can replicate the leak issue? This way we'll also know that we fixed the issue across all supported Node.js versions we work on.

It doesn't need to be your real code, just code I can copy and paste and run to see the leak. I can then point autocannon at the server code you provide and watch the v8 memory output to monitor the leak and object dump to see what is getting leaked 👍

from timeout.

nkbt avatar nkbt commented on August 15, 2024

Due to how the gc works there shouldn't be a memory leak

What do you mean by that? I am pretty sure when I subscribe to events and not unsubscribe afterwards, that callback will be hanging around forever.
I reckon there could be potentially some internal req destroy that will unsub from all events when request completes, but I haven't check nodejs source for this yet.
Maybe I'm missing something?

In my internal test (on actual app) as soon as I switched respond: false and implemented my own solution that looked very much like

const timeout = (req, res, next) => connectTimeout(req, res, () => {
  const onTimeout = delay => {
    req.removeListener('timeout', onTimeout);
    next(new TimeoutError())
  };
  req.on('timeout', onTimeout);
  next();
})

After 200k requests container was using 150mb of mem.
With const timeout = connectTimeout after same 200k requests it was about 175mb.
Note, that I was testing it inside fresh docker container each time, so env is absolutely clear at the start and there was nothing else happening between runs.

I may profile it a bit more on Monday, but as a rule of thumb I'd always unsub from any subscription

PS: it is not that easy to test since request is quite small. Though I guess if add there some very large string var in closure it would be way more obvious.

from timeout.

nkbt avatar nkbt commented on August 15, 2024

Probably if we check connectTimeout in isolation without express and stuff and then test - it would be more obvious. Because when I was checking mem dumps it had quite a few diffs =( and I did not have enough time to investigate it further. I found interesting that no matter how many requests I made and if I ran gc manually all the time, there always was last (exactly last) route leftover in diffs (strings)

from timeout.

dougwilson avatar dougwilson commented on August 15, 2024

Haha, yea, they can be tedious. I've looked through so many I have some shell scripts to help me expediate some tasks, which is why I'm happy to volunteer to figure it out 👍

from timeout.

nkbt avatar nkbt commented on August 15, 2024

Actually req.on can be just replaced with req.once('timeout')
https://nodejs.org/api/events.html#events_emitter_once_eventname_listener

from timeout.

dougwilson avatar dougwilson commented on August 15, 2024

So I'm going to close this since it seems to have stalled and never got the needed information here to proceed.

from timeout.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.