Very high CPU usage using pnpm about nollup HOT 21 CLOSED

There's definitely something wrong with pnpm and chokidar. While I can't fully replicate the same issue, when I use pnpm I get extremely long build times and high CPU usage. Commenting out chokidar solves the problem. I found a couple of issues that probably hint at what's going on:

paulmillr/chokidar#773
pnpm/pnpm#119

Can you try this change directly in your node_modules in the dev middleware:

let watcher = chokidar.watch(options.watch || process.cwd(), {
    // ignored:  ['**/node_modules/**/*', '**/.git/**/*'],
    ignored: (path => path.includes('node_modules') || path.includes('.git'))
});

from nollup.

You're welcome, thanks for your ideas!

It's purely to get some sync of style, navigation and scroll on all my devices (mobile, tablet, desktop) when developing for responsive and stuff, it's very handy. I used to have it along the webpack-dev-server and worked like a charm.

from nollup.

Great! I should hopefully have a fix for the first issue by tomorrow. :)

from nollup.

Looks good to me! Thanks for the fix.

from nollup.

Hey @flayks!

If no error is being thrown in the terminal, it means that the bundle should be accessible.

What happens if you open http://localhost:3000? Is there an index.html file being served? I noticed in your dev command that ---content-base isn't pointing to anything. Are you able to open http://localhost:3000/main.[hash].js?

CPU usage is very concerning. My initial guess is that there's potentially an infinite loop happening when parsing the files. Is there anything note-worthy or unusual in the code like circular dependencies or any big libraries that you're using?

from nollup.

I made some tweaks to follow pretty much exactly what the examples are (public folder with an index.html including the script to /main.js file), but it seems that I have the same issues, 440% CPU.

This way, the page loads and I have a result but the fans of my laptop are not happy about it haha

"scripts": {
   "dev": "cross-env NODE_ENV=development nollup -c nollup.config.js --content-base public --port 3000 --hot"
}

I also changed my rollup config to minimum in order to track issues but that's pretty much the best I could do I'm afraid

import resolve from '@rollup/plugin-node-resolve'
import commonjs from '@rollup/plugin-commonjs'

// Rollup config
export default {
	input: './src/main.js',
	output: {
		dir: 'dist',
		format: 'esm',
		entryFileNames: '[name].js'
	},

	// Plugins
	plugins: [
		// Javascript
		resolve({
			browser: true
		})
	]
}

Terminal output:

> npm run dev

> [email protected] dev /Users/user/Desktop/Rollup
> cross-env NODE_ENV=development nollup -c nollup.config.js --content-base public --port 3000 --hot

Listening on http://localhost:3000
Compiled in 2ms.
Compiled in 1ms.
Compiled in 0ms.
Compiled in 1ms.
Compiled in 0ms.
Compiled in 0ms.
^C%

from nollup.

Do the examples also cause the CPU to spike to 440% or just the project you're trying to use Nollup in?

Would it be possible to send a minimal reproducible example that triggers the high CPU usage?

from nollup.

Do the examples also cause the CPU to spike to 440% or just the project you're trying to use Nollup in?

Would it be possible to send a minimal reproducible example that triggers the high CPU usage?

It seems that, at least the nollup/examples/example-react-hot-loader example works without CPU usage which means that it might come from my setup, but I hardly see why it could be that way considering how minimal my setup is :/ Is it about package versions or something?

from nollup.

I'm not seeing anything obvious that would be the culprit. As for package versions, it seems unlikely. Are there directories (excluding node_modules) that contain a huge number of files? If you replace the plugins with the ones that the example uses, does that solve the problem?

from nollup.

I'm not seeing anything obvious that would be the culprit. As for package versions, it seems unlikely. Are there directories (excluding node_modules) that contain a huge number of files? If you replace the plugins with the ones that the example uses, does that solve the problem?

Alright, so I did some other tests.

1. Copy an example
   Apart from copying the example-dynamic-import and using nollup from the npm package instead of the cli file, I still have some serious CPU issues. That's all I changed in the package.json script.
2. Run the example from the repo
   It all runs pretty smoothly, almost no CPU usage, just works. I just had to install some express/chokidar dependencies in order to make it work but that's pretty much it

I don't really know what could be the issue here but there is apparently one. Maybe my node environment? I use node v13.7.0 via n (homebrew - macOS 10.15.2) 🤷

from nollup.

Thanks for the update!

I tried to replicate by copying the example and installing Nollup from the package using Node 13.7.0, but still not able to reproduce. Dependencies should be all of their latest as well. I'm using Windows so I can't confirm if it's a Mac specific thing.

I'm not sure what else it could be. Are they running in the same directory on your system both the copy and the repo?

from nollup.

Hm that is strange! I do, they are both in my ~/Desktop folder. Would it affect the performance and usage if it's anywhere else? I just feel running the nollup command as an npm package is causing the trouble here

from nollup.

There shouldn't be any performance drop based on where it is, but it could affect chokidar potentially if there's a lot of files being watched, but it doesn't sound like that is the case if they're both in the same location. The npm package doesn't go through any compilation, and would be the same as running the code directly from the repo which makes the whole thing even weirder.

Can you try comment out lines 117 to 143 in nollup/dev-middleware.js? https://github.com/PepsRyuu/nollup/blob/master/lib/dev-middleware.js#L117

from nollup.

I think I found the bad guy, it's pnpm. I use it over npm or yarn because it allows you to symlink already downloaded node modules to your project and then work offline or setup your modules faster on top of saving lots of disk space.

I think Chokidar/some library doesn't really like that boy, it worked smoothly without CPU usage with npm and yarn. So I suppose it's not really a nollup concern, more of a third party module used by nollup? What do you think?

from nollup.

That said I haven't found any issue mentioning pnpm on the Chokidar github project, so it might be something else?

from nollup.

Yoohoo! I think we have a winner here 🎉
That totally fixes the problem. I guess they changed their ignored file checking in the meantime?

I still have some issues with BrowserSync but this is something else 🙈

WebSocket connection to 'ws://localhost:3000/__hmr' failed: Connection closed before receiving a handshake response
> (anonymous) | @ | main.[hash].js:174
> (anonymous) | @ | main.[hash].js:316
> (anonymous) | @ | main.[hash].js:322

const nollup = process.env.NOLLUP
const port = (process.argv[process.argv.indexOf('--port') + 1]) || 3000
...
// BrowerSync
nollup && browsersync({
    port: 3000,
    proxy: 'http://localhost:' + port,
    open: false,
    notify: false
}),

from nollup.

Thanks for testing! I'll have a look into this in more detail later this evening, just to make sure there's no consequences in terms of general performance and no oversights that might break compatibility with existing projects.

Not sure I understand why browsersync is being used. That could cause a conflict because Nollup is already running a web server. Is there a use case I'm missing?

from nollup.

Ah I see, first I'm hearing about it. 😅 Sounds interesting!

From your config, it looks like they're running on the same port?

from nollup.

It is very cool yea.

No actually, I run nollup on port 3100 (3000 by default), and then proxy that server on the BrowerSync one. I'd say it's more of a proxy issue maybe? But could be related to nollup, no idea.

from nollup.

Okay, this sounds like fixing the issue with BrowserSync (so def not nollup related):

// BrowserSync
nollup && browsersync({
    port: bsPort,
    proxy: {
        target: 'http://localhost:' + port,
        ws: true
    },
    open: false,
    notify: false
}),

from nollup.

Released 0.10.3. Let me know if it works for you!

from nollup.