Yarn PnP compatibility? about import-meta-resolve HOT 18 CLOSED

I don’t particularly mean that remark itself is esm. More that this project loads files the way Node.js loads files, when in ESM. It can load CJS fine. But yarn PNP is doing magic things to hook into require calls, to not need ‘node_modules’, those hooks are different on the ESM side

from import-meta-resolve.

I'm gonna close this out. Thanks @wooorm !

from import-meta-resolve.

Perhaps that should just be listed as a limitation of this package.

It is, readme says no loaders & no env variables, no lots of things. There are tons of (experimental) env variable APIs that change how resolving happens.

Those are interesting, but there are also cases where you (as a package author) don’t want the user’s configuration to change what you resolve. That’s my primary use case. And I believe most people who currently use this’s use case too.

This package also allows to pass conditions for example (and doesn’t inherit the users conditions). To illustrate, it could be used to find a browser version first, and fall back otherwise. Such use cases would not work if the users conditions are used.

So if loaders were to be supported, they should be configurable. Inferring the current loaders/conditions etc could be added too.
But this is sooo much work (also to maintain, it’s lots of code that changes often and has to manually be pulled in). I’m not interested.

That would address the issue.

Then there would be significant differences between when the actual import.meta.resolve is available and when not. As in, Yarn PnP wouldn’t work on old Node. At that point, just use import.meta.resolve?

from import-meta-resolve.

Can you make a tiny example repo that showcases this, using a modern yarn? I am not a yarn (pnp) user myself. That would likely help me.

from import-meta-resolve.

@wooorm For sure - thank you for looking into this.

Here's a minimal reproduction (and the README.md describes reproduction steps) https://github.com/bencmbrook/import-meta-resolve-pnp-repro

from import-meta-resolve.

Thanks. Running yarn in that repo does generate node_modules. How do you expect me to install dependencies without node_modules?

from import-meta-resolve.

Hm, must be some environment differences - let me dig in on that.

Can you confirm yarn -v logs 3.3.0?

from import-meta-resolve.

Yes, 3.3.0. Node 19 or 16.15, then running yarn, both generate a node_modules.

from import-meta-resolve.

Managed to reproduce. It does say when running yarn:

➤ YN0000: │ ESM support for PnP uses the experimental loader API and is therefore experimental

That’s likely it. yarn doesn’t support ESM well.

from import-meta-resolve.

Hmm yes that could be it, but I don't think it's because the target package, remark, is ESM. I tested it with a package import that's CJS (swap remark for request) and I get the same error.

from import-meta-resolve.

Ah, yeah I'm tracking.

I just pushed up another test, using the native import.meta.resolve() and it resolves successfully.

If you pull the changes, yarn start runs the script with --experimental-import-meta-resolve

from import-meta-resolve.

Digging deeper here... The source of the ERR_MODULE_NOT_FOUND is here

Oddly, the reference implementation also does this? https://github.com/nodejs/node/blob/3c040348fe9c49687a796fbeea3fe7a9bc1251ed/lib/internal/modules/esm/resolve.js#L874-L875

This might be Yarn's magic? https://github.com/yarnpkg/berry/blob/master/packages/yarnpkg-pnp/sources/esm-loader/hooks/resolve.ts

from import-meta-resolve.

There may have landed new code to support ESM loaders in the upstream import.meta.resolve algorithm.

I am wondering if Yarn PnP can be solved in your actual use case though.
You will need to call Node.js (or even somehow this package deep down through load-plugin in unified-engine through remark-language-server) with an experimental Node.js flag. And pass Yarns custom loader in. Where would you even do that?

Here’s what Yarn says about Yarn + ESM. My conclusion is different though: Don‘t use Yarn if it doesn’t support ESM/Node’s resolving algorithm.

from import-meta-resolve.

As I understand it (I backported recent changes just now), loaders are implemented above resolve: https://github.com/nodejs/node/blob/3ebe7535b456771078368ba72345f8d2e16ce695/lib/internal/modules/esm/loader.js.

I don’t think there’s anything to do in this project.

from import-meta-resolve.

Yeah, I think I'm just out of luck with this combo. Thanks again for looking into this.

I am wondering if Yarn PnP can be solved in your actual use case though.

My use-case is specifically getting the remark VSCode extension working in a PnP repo, so I have little control over the Node flags. But I might be able to yarn patch this library (or something else on the load-plugin, unified-engine, remark-language-server chain). Maybe a patch to switch load-plugin to use require.resolve or something.

from import-meta-resolve.

As I understand it (I backported recent changes just now), loaders are implemented above resolve: https://github.com/nodejs/node/blob/3ebe7535b456771078368ba72345f8d2e16ce695/lib/internal/modules/esm/loader.js.

It's the other way around - import.meta.resolve must go through loaders. If it wasn't, calling import(import.meta.resolve(...)) wouldn't work, which would be unacceptable. You can easily check it:

import {register} from 'module';

// export async function resolve() {
//   return {shortCircuit: true, url: 'file:///lol'};
// }
register('data:application/javascript;charset=utf-8,export%20async%20function%20resolve%28%29%20%7B%0D%0A%20%20return%20%7BshortCircuit%3A%20true%2C%20url%3A%20%27file%3A%2F%2F%2Flol%27%7D%3B%0D%0A%7D');

console.log(import.meta.resolve('foo'));

Prints

file:///hello

In other words, import-meta-resolve should implement loader support to be on parity with import.meta.resolve, this isn't an issue with PnP itself.

from import-meta-resolve.

It's the other way around - import.meta.resolve must go through loaders

Then where is the code in Node that does that?

I know that it’s still done by the function at import.meta.resolve.
But it’s not in the code Node uses to resolve things.
Similar to how import.meta.url can’t be polyfilled.

In other words, import-meta-resolve should implement loader support to be on parity with import.meta.resolve

Where is the API in Node to find which loaders are currently turned on? I don’t think it exists. I don’t think this can be done.

from import-meta-resolve.

Where is the API in Node to find which loaders are currently turned on? I don’t think it exists. I don’t think this can be done.

That's tricky indeed - Node.js recently changed the way hooks are registered, so you can't even rely on extracting the --loader values from process.execArgv. Perhaps that should just be listed as a limitation of this package.

With that said, shouldn't the ponyfill delegate to the real import.meta.resolve when running on Node.js versions that expose it unflagged, ie 18.19+ and 20.6+? That would address the issue.

from import-meta-resolve.