Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory about pnpm HOT 26 CLOSED

@sammarks put your observations to the existing issue about the same error.

from pnpm.

Should be fixed by #8060

from pnpm.

🚢 9.1.1

from pnpm.

@zkochan this issue still persists even after using pnpm version 9.1.1

from pnpm.

How can I reproduce this? Is the project open source?

from pnpm.

@zkochan it is not opensource, im using nx monorepo with node version 18.16.0 & pnpm version 9.1.1
i will try to come with some sample project if possible

from pnpm.

FWIW I am unfortunately not able to provide the source for my example either, but I have been doing a bit of debugging on my end.

On my machine, the process doesn't get to the point of running out of memory, but does peg the CPU.

In my case, inside resolvePeersOfChildren, calculateDepPaths has a length of 92 for one specific package (I'm guessing the package itself is irrelevant as it depends on the packages that depend on it), which seems a lot longer than the rest. That calls analyzeGraph from graph-cycles with 156 nodes, and ~a quarter of those nodes having ~20-30 dependencies. The call to analyzeGraph does eventually finish without running out of memory, but takes several minutes to complete.

I verified this by installing v1.2.1 of graph-cycles separately and passing it the same payload.

Not sure if there's a way to reduce the size of the graph passed to graph-cycles, or if there's a way to improve the performance of graph-cycles.

Our environment is a large monorepo with ~200 interconnected packages, with a lot of them referencing the others as peer dependencies (version specifier "<1.0.0" for all of the interrelated peers). The installation inside the main monorepo succeeds without any slowness, but running an installation inside another much smaller monorepo that depends on a lot of the packages from the larger monorepo is the one that stalls. The packages from the main monorepo are being downloaded from a registry instead of any local workspace linking.

So I would imagine this might be reproducible by creating a smaller monorepo (5 packages) that depend on other packages with a lot of peer dependencies to drive the size of that graph up.

from pnpm.

Chiming in to say we're experiencing this using github actions on a turborepo monorepo building a sveltekit app on pnpm 9.1.1.

✓ 1789 modules transformed.
rendering chunks...
<--- Last few GCs --->
[5527:0x661a640] 68118 ms: Mark-Compact (reduce) 2035.4 (2076.3) -> 2035.3 (2076.8) MB, 1244.63 / 0.02 ms (+ 84.4 ms in 7 steps since start of marking, biggest step 26.4 ms, walltime since start of marking 1353 ms) (average mu = 0.300, current mu = 0.[5527:0x661a640] 69962 ms: Mark-Compact (reduce) 2041.8 (2082.5) -> 2040.6 (2082.4) MB, 1834.44 / 0.00 ms (average mu = 0.153, current mu = 0.006) allocation failure; scavenge might not succeed
<--- JS stacktrace --->
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----
1: 0xb84bd6 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [node]
2: 0xefec30 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
3: 0xefef17 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
4: 0x1110925 [node]
5: 0x11287a8 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
6: 0x10fe8c1 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
7: 0x10ffa55 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
8: 0x10dd0a6 v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
9: 0x1537e36 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
10: 0x7fb92ce99ef6
Aborted (core dumped)
 ELIFECYCLE  Command failed with exit code 134.

from pnpm.

Just a follow up to this as I continue to debug. Here is my github actions file. I tried not building through turbo and going directly to build within the app directory.

jobs:
build:
name: Build and Test
timeout-minutes: 15
runs-on: ubuntu-latest

steps:
  - name: Check out code
    uses: actions/checkout@v4
    with:
      fetch-depth: 2

  - uses: pnpm/action-setup@v4
    with:
      version: 9.1.1

  - name: Setup Node.js environment
    uses: actions/setup-node@v4
    with:
      node-version: 20
      cache: 'pnpm'

  - name: Install dependencies
    run: pnpm install --no-frozen-lockfile

  - name: Install playwright browsers
    run: pnpx playwright install --with-deps

  - name: Build
    working-directory: ./apps/web
    run: pnpm build 

  - name: Test
    working-directory: ./apps/web
    run: pnpm test

Still resulted in:
<--- JS stacktrace --->
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----
1: 0xb84bd6 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [node]
2: 0xefec30 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
3: 0xefef17 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
4: 0x1110925 [node]
5: 0x11287a8 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
6: 0x10fe8c1 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
7: 0x10ffa55 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
8: 0x10dd0a6 v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
9: 0x1537fe1 v8::internal::Runtime_AllocateInOldGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
10: 0x7fe059e99ef6
Aborted (core dumped)
 ELIFECYCLE  Command failed with exit code 134.

from pnpm.

I have reduced memory consumption in this PR: #8084

I don't know if it will be enough for you all. Maybe you could try to build pnpm from that branch and see if it helps.

from pnpm.

Try 9.1.2

from pnpm.

@zkochan i'm still facing this issue, happens on bitbucket pipeline
sadly i can't provide sample code for this

- step: name: Trigger pnpm build test size: 2x image: node:18.16.0 script: - corepack enable - corepack prepare pnpm@latest - pnpm install - npm run build:manager caches: - pnpm

> nx run manager-dashboard:build:prod vite v4.3.9 building for production... transforming... replace "apps/manager-dashboard/src/environments/environment.ts" with "apps/manager-dashboard/src/environments/environment.prod.ts" replace "apps/manager-dashboard/src/environments/environment.ts" with "apps/manager-dashboard/src/environments/environment.prod.ts" replace "apps/manager-dashboard/src/environments/environment.ts" with "apps/manager-dashboard/src/environments/environment.prod.ts" replace "apps/manager-dashboard/src/environments/environment.ts" with "apps/manager-dashboard/src/environments/environment.prod.ts" replace "apps/manager-dashboard/src/environments/environment.ts" with "apps/manager-dashboard/src/environments/environment.prod.ts" replace "apps/manager-dashboard/src/environments/environment.ts" with "apps/manager-dashboard/src/environments/environment.prod.ts" replace "apps/manager-dashboard/src/environments/environment.ts" with "apps/manager-dashboard/src/environments/environment.prod.ts" node_modules/.pnpm/[email protected]/node_modules/pdfjs-dist/build/pdf.js (1982:23) Use of eval in "node_modules/.pnpm/[email protected]/node_modules/pdfjs-dist/build/pdf.js" is strongly discouraged as it poses security risks and may cause issues with minification. ✓ 7428 modules transformed. rendering chunks... <--- Last few GCs ---> [266:0x639cd20] 156539 ms: Mark-sweep 1942.0 (2101.7) -> 1917.0 (2094.2) MB, 1895.3 / 0.0 ms (average mu = 0.735, current mu = 0.547) allocation failure; scavenge might not succeed [266:0x639cd20] 158585 ms: Mark-sweep 1951.1 (2128.3) -> 1951.1 (2122.8) MB, 1974.5 / 0.0 ms (average mu = 0.552, current mu = 0.035) allocation failure; scavenge might not succeed <--- JS stacktrace ---> FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory 1: 0xb7a940 node::Abort() [/usr/local/bin/node] 2: 0xa8e823 [/usr/local/bin/node] 3: 0xd5c940 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node] 4: 0xd5cce7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node] 5: 0xf3a3e5 [/usr/local/bin/node] 6: 0xf4c8cd v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node] 7: 0xf26fce v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/bin/node] 8: 0xf28397 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/bin/node] 9: 0xf088e0 v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [/usr/local/bin/node] 10: 0xf00354 v8::internal::FactoryBase<v8::internal::Factory>::AllocateRawWithImmortalMap(int, v8::internal::AllocationType, v8::internal::Map, v8::internal::AllocationAlignment) [/usr/local/bin/node] 11: 0xf02608 v8::internal::FactoryBase<v8::internal::Factory>::NewRawOneByteString(int, v8::internal::AllocationType) [/usr/local/bin/node] 12: 0x1042959 v8::internal::JsonParser<unsigned short>::MakeString(v8::internal::JsonString const&, v8::internal::Handle<v8::internal::String>) [/usr/local/bin/node] 13: 0x1044676 v8::internal::JsonParser<unsigned short>::ParseJsonValue() [/usr/local/bin/node] 14: 0x104515f v8::internal::JsonParser<unsigned short>::ParseJson() [/usr/local/bin/node] 15: 0xde2523 v8::internal::Builtin_JsonParse(int, unsigned long*, v8::internal::Isolate*) [/usr/local/bin/node] 16: 0x16fb7b9 [/usr/local/bin/node]

from pnpm.

I believe this should do it: #8106

from pnpm.

Did one more memory optimization: #8109

I believe most of the issues will be solved in the next release.

from pnpm.

Try 9.1.3

from pnpm.

@zkochan still same :(

from pnpm.

I think I have found the core issue. I will try to fix it.

from pnpm.

As a temporary workaround, you should be able to add these settings to your .npmrc in the root of your project:

dedupe-peer-dependents=false
auto-install-peers=false

from pnpm.

Try v9.1.4

from pnpm.

Just tried it with 9.1.4 still getting this

<--- Last few GCs --->
[4770:0x75e72e0] 74721 ms: Scavenge (reduce) 2036.0 (2079.8) -> 2035.9 (2080.8) MB, 14.05 / 0.00 ms (average mu = 0.300, current mu = 0.275) allocation failure;
[4770:0x75e72e0] 76464 ms: Mark-Compact (reduce) 2037.8 (2082.1) -> 2037.2 (2082.2) MB, 1346.10 / 0.00 ms (average mu = 0.274, current mu = 0.248) allocation failure; GC in old space requested
<--- JS stacktrace --->
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----
1: 0xb82c28 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [node]
2: 0xeed540 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
3: 0xeed827 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
4: 0x10ff3c5 [node]
5: 0x1117248 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
6: 0x10ed361 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
7: 0x10ee4f5 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
8: 0x10cbb46 v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
9: 0x1527b21 v8::internal::Runtime_AllocateInOldGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
10: 0x7f8d34e99ef6
Aborted (core dumped)
 ELIFECYCLE  Command failed with exit code 134.

from pnpm.

The projects that reproduced the issue for me work now. If you have sharable examples that reproduce the issue, please attach them.

from pnpm.

Just got the opportunity to check it again in my environment; in my case the fix in 9.1.3 appears to have fixed it.

However, now I get a new error after the install is complete and dependencies are built:

I get this same error with 9.1.4, so ignore the 9.1.3 in the path in the error report. I've upgraded to 9.1.4 and tried again to find the same issue.

pnpm: Invalid string length
    at writeBlockMapping (/Users/sammarks/Library/pnpm/global/5/.pnpm/[email protected]/node_modules/pnpm/dist/pnpm.cjs:16817:20)
    at writeNode (/Users/sammarks/Library/pnpm/global/5/.pnpm/[email protected]/node_modules/pnpm/dist/pnpm.cjs:16883:13)
    at writeBlockMapping (/Users/sammarks/Library/pnpm/global/5/.pnpm/[email protected]/node_modules/pnpm/dist/pnpm.cjs:16808:14)
    at writeNode (/Users/sammarks/Library/pnpm/global/5/.pnpm/[email protected]/node_modules/pnpm/dist/pnpm.cjs:16883:13)
    at Object.dump (/Users/sammarks/Library/pnpm/global/5/.pnpm/[email protected]/node_modules/pnpm/dist/pnpm.cjs:16977:11)
    at yamlStringify (/Users/sammarks/Library/pnpm/global/5/.pnpm/[email protected]/node_modules/pnpm/dist/pnpm.cjs:114456:32)
    at writeLockfiles (/Users/sammarks/Library/pnpm/global/5/.pnpm/[email protected]/node_modules/pnpm/dist/pnpm.cjs:114470:23)
    at async Promise.all (index 0)
    at async _installInContext (/Users/sammarks/Library/pnpm/global/5/.pnpm/[email protected]/node_modules/pnpm/dist/pnpm.cjs:192699:9)
    at async installInContext (/Users/sammarks/Library/pnpm/global/5/.pnpm/[email protected]/node_modules/pnpm/dist/pnpm.cjs:192812:16)

Modified that bit of code in the 9.1.4 transpiled JS to be:

} else {
  pairBuffer += ": ";
}
pairBuffer += state.dump;
try {
  _result += pairBuffer;
} catch (err) {
  console.error('would have run into a problem here')
  console.info(_result.length)
  console.info(pairBuffer.length)
  process.exit(1)
}

And that outputs:

would have run into a problem here
19313147
533455214

So looks like we're running into some very large strings here. Even if it were able to handle that, it seems like the lockfile would be a few hundred MB in size.

That was with:

auto-install-peers=false
dedupe-peer-dependents=true

Running it with:

auto-install-peers=false
dedupe-peer-dependents=false

results in the same error, but it seems to happen later in the process. This time here's the output I get:

would have run into a problem here
526644599
77564213

Happy to help debug some more if you tell me what variables you want to see, but unfortunately I'm not able to provide access to the specific repository where it's happening.

If it helps here's my full config:

link-workspace-packages=deep
strict-peer-dependencies=false

auto-install-peers=false
resolution-mode=lowest-direct
dedupe-peer-dependents=false
disallow-workspace-cycles=true
resolve-peers-from-workspace-root=false

I can also make a separate issue for this or participate in the existing thread to keep things organized; just let me know.

from pnpm.

Sorry I know this isn't very helpful, I've been offline, but tried again today with 9.3.0 and I'm still getting the error. The project that's attempting to be built is a sveltekit app inside a turbo repo mono repo. Will try to dive in more soon. For now I'm not building on CI and only running the tests.

✓ 1926 modules transformed.
rendering chunks...

<--- Last few GCs --->

[4834:0x59665f0] 68976 ms: Scavenge 2040.3 (2083.2) -> 2040.3 (2084.2) MB, 4.23 / 0.00 ms (average mu = 0.356, current mu = 0.364) allocation failure;

<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

1: 0xb82c78 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [node]
2: 0xeefa80 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
3: 0xeefd67 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
4: 0x1101905 [node]
5: 0x1101e94 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [node]
6: 0x1118d84 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::internal::GarbageCollectionReason, char const*) [node]
7: 0x111959c v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
8: 0x10ef8a1 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
9: 0x10f0a35 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
10: 0x10cd156 v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [node]
11: 0x10bed84 v8::internal::FactoryBasev8::internal::Factory::AllocateRawWithImmortalMap(int, v8::internal::AllocationType, v8::internal::Map, v8::internal::AllocationAlignment) [node]
12: 0x10c1566 v8::internal::FactoryBasev8::internal::Factory::NewRawOneByteString(int, v8::internal::AllocationType) [node]
13: 0x10d8584 v8::internal::Factory::NewStringFromUtf8(v8::base::Vector const&, v8::internal::AllocationType) [node]
14: 0xf02312 v8::String::NewFromUtf8(v8::Isolate*, char const*, v8::NewStringType, int) [node]
15: 0xde09e1 [node]
16: 0xc116b4 node::encoding_binding::BindingData::DecodeUTF8(v8::FunctionCallbackInfov8::Value const&) [node]
17: 0x7f8fffe0d546
Aborted (core dumped)
 ELIFECYCLE  Command failed with exit code 134.

from pnpm.

But this happens on build, right? Not during installation. In that case it isn't related to pnpm.

from pnpm.

Correct on build. Yesterday I updated to Turbo 2.0, cleaned up some packages and reintroduced the build step on CI and it's working, so makes sense that it isn't related to pnpm.

from pnpm.

@bhavesh2510 please check again with pnpm v9.3

from pnpm.