Git Product home page Git Product logo

loaders's Introduction

Loaders Team

Purpose

The Node.js Loaders Team maintains and actively develops the ECMAScript Modules Loaders implementation in Node.js core.

History

This team is spun off from the Modules team. We aim to implement the use cases that went unfulfilled by the initial ES modules implementation that can be achieved via loaders.

Project

Status

Milestone 1: Parity with CommonJS

Before extending into new frontiers, we need to improve the loaders API enough that users can do just about everything they could do in CommonJS with ESM + loaders. (Outside of loaders scope, but related to the goal of parity between CommonJS and ESM, is finishing and stabilizing --experimental-vm-modules.)

Milestone 2: Stability

Milestone 3: Usability improvements

  • Provide a way to register loaders via configuration, for example via adding support for .env files to Node.js or having node read configuration from a new field in package.json or other configuration file. See nodejs/node#46826, #98, nodejs/node#43973 (comment). nodejs/node#48890.

  • Integrated support for external formats.

    • Phase 1: Support identifying external modules (eg typescript); see nodejs/node#49704.
    • Phase 2: Support guided remediation via package manager search (eg npm search … typescript).
    • Phase 3: Automatically configure Node.
  • First-class support for import maps that doesn’t require a custom loader.

  • Add helper/utility functions to reduce boilerplate in user-defined hooks.

    • Start with helpers for retrieving the closest parent package.json associated with a specifier string; and for retrieving the package.json for a particular package by name (which is not necessarily the same result).

    • Potentially include all the functions that make up the ESM resolution algorithm as defined in the spec. Create helper functions for each of the functions defined in that psuedocode: esmResolve, packageImportsResolve, packageResolve, esmFileFormat, packageSelfResolve, readPackageJson, packageExportsResolve, lookupPackageScope, packageTargetResolve, packageImportsExportsResolve, patternKeyCompare. (Not necessarily all with these exact names, but corresponding to these functions from the spec.)

    • Follow up with similar helper functions that make up what happens within Node’s internal load. (Definitions to come.)

  • Helper/utility functions to allow access to the CommonJS named exports discovery algorithm (cjs-module-lexer).

  • Hooks for customizing the REPL, including transpilation and tab completion. Support users pasting TypeScript (or CoffeeScript or whatever) into the REPL and having just as good an experience as with plain JavaScript.

    • Support top-level await in the REPL API, if possible.
  • Allow customizing string inputs: --eval CLI flag, Worker constructor, stdin, etc.

  • Hooks for customizing the stack trace (in other words, a hook version of Error.prepareStackTrace). This would allow transpiled languages to improve the output.

  • Hooks for customizing filesystem calls, for allowing things like virtual filesystems or archives treated as volumes.

  • Inherit configuration blob to worker threads and child processes.

loaders's People

Contributors

arcanis avatar brev avatar coderaiser avatar cspotcode avatar d3x0r avatar dereknongeneric avatar geoffreybooth avatar giltayar avatar jakobjingleheimer avatar privatenumber avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

loaders's Issues

Import assertions with loaders

Here's a draft of what solution I can think of regarding relations between loaders and import assertions.

Relation of ESMLoader class with import assertions:

Before resolve() resolve() Between resolve() and load() load() After load()
Check if importAssertions.type value is supported nothing use importAssertions.type to check for race conditions nothing check if finalFormat is compatible with importAssertions.type
nothing pass importAssertions to resolve() check if importAssertions.type value is supported, and use it to check for race conditions nothing check if finalFormat is compatible with importAssertions.type
nothing pass importAssertions to resolve() disallow race conditions?1 nothing check if finalFormat is compatible with importAssertions.type
nothing pass importAssertions to resolve() one of the above pass importAssertions to load() check if importAssertions.type value is supported and compatible with finalFormat
nothing pass importAssertions to resolve() one of the above pass importAssertions to load() and let load() do the checks nothing
nothing nothing use importAssertions.type to check for race conditions pass importAssertions to load() and let load() do the checks nothing
nothing nothing nothing pass importAssertions to load() and let load() do the checks nothing
  1. The first one is the approach on my first Unflag import assertions PR, drawbacks of this approach have been discussed.
  2. The second seems really appealing to me, it would allow folks to write hooks to support assertion-less JSON, and also support import assertions that are not supported by Node.js.
  3. Same thing except the loader throws if a URL is loaded with different assertions. It puts the burden of handling race conditions to the resolve hook. Not sure this is actually doable and/or desirable.
  4. pass the output importAssertions of resolve() to load() context. This adds complexity, not sure it's worth it.
  5. Same but now the loader hook is in charge of doing the verification, so users can opt-out. That seems... dangerous? but maybe it's actually OK?
  6. Same but import assertions are not passed to resolve().

My (new) favourite is 2.: it seems to me like the perfect balance between simplicity and letting users do powerful things (e.g.: it would be quite easy to implement a loader that supports assertionless imports). I have a PR almost ready implementing this.

Anyway, we can discuss that on next week meeting since Goeffrey invited me, but happy to hear thoughts before that if people want to share them.

Footnotes

  1. if the resolver returns an url that's already in the cache and the returned importAssertion.type has a value different from the one in the cache, throw a TypeError.

Instrumenting fetching algorithm parts

the thread of https://twitter.com/bradleymeck/status/1454093832539320325 pointed out with the consolidation of hooks that the provided during the fetching of a resource body from a URL no longer is automated by the nature of the hooks.

this somewhat mirrors Yarn (CC: @arcanis ) and its desire to have node automate parts of the URL operations for things.

Likely we should investigate how to instrument this. Likely 2 sets of hooks that are available one for instrumenting the resolver in the node algorithm and one for the actual fetching operations.

Node.js Loaders Team Meeting 2022-02-15

Time

UTC Tue 15-Feb-2022 19:00 (07:00 PM):

Timezone Date/Time
US / Pacific Tue 15-Feb-2022 11:00 (11:00 AM)
US / Mountain Tue 15-Feb-2022 12:00 (12:00 PM)
US / Central Tue 15-Feb-2022 13:00 (01:00 PM)
US / Eastern Tue 15-Feb-2022 14:00 (02:00 PM)
EU / Western Tue 15-Feb-2022 19:00 (07:00 PM)
EU / Central Tue 15-Feb-2022 20:00 (08:00 PM)
EU / Eastern Tue 15-Feb-2022 21:00 (09:00 PM)
Moscow Tue 15-Feb-2022 22:00 (10:00 PM)
Chennai Wed 16-Feb-2022 00:30 (12:30 AM)
Hangzhou Wed 16-Feb-2022 03:00 (03:00 AM)
Tokyo Wed 16-Feb-2022 04:00 (04:00 AM)
Sydney Wed 16-Feb-2022 06:00 (06:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/loaders

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2022-04-26

Time

UTC Tue 26-Apr-2022 18:00 (06:00 PM):

Timezone Date/Time
US / Pacific Tue 26-Apr-2022 11:00 (11:00 AM)
US / Mountain Tue 26-Apr-2022 12:00 (12:00 PM)
US / Central Tue 26-Apr-2022 13:00 (01:00 PM)
US / Eastern Tue 26-Apr-2022 14:00 (02:00 PM)
EU / Western Tue 26-Apr-2022 19:00 (07:00 PM)
EU / Central Tue 26-Apr-2022 20:00 (08:00 PM)
EU / Eastern Tue 26-Apr-2022 21:00 (09:00 PM)
Moscow Tue 26-Apr-2022 21:00 (09:00 PM)
Chennai Tue 26-Apr-2022 23:30 (11:30 PM)
Hangzhou Wed 27-Apr-2022 02:00 (02:00 AM)
Tokyo Wed 27-Apr-2022 03:00 (03:00 AM)
Sydney Wed 27-Apr-2022 04:00 (04:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2021-11-09

Time

UTC Tue 09-Nov-2021 18:00 (06:00 PM):

Timezone Date/Time
US / Pacific Tue 09-Nov-2021 10:00 (10:00 AM)
US / Mountain Tue 09-Nov-2021 11:00 (11:00 AM)
US / Central Tue 09-Nov-2021 12:00 (12:00 PM)
US / Eastern Tue 09-Nov-2021 13:00 (01:00 PM)
EU / Western Tue 09-Nov-2021 18:00 (06:00 PM)
EU / Central Tue 09-Nov-2021 19:00 (07:00 PM)
EU / Eastern Tue 09-Nov-2021 20:00 (08:00 PM)
Moscow Tue 09-Nov-2021 21:00 (09:00 PM)
Chennai Tue 09-Nov-2021 23:30 (11:30 PM)
Hangzhou Wed 10-Nov-2021 02:00 (02:00 AM)
Tokyo Wed 10-Nov-2021 03:00 (03:00 AM)
Sydney Wed 10-Nov-2021 05:00 (05:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2021-12-21

Time

UTC Tue 21-Dec-2021 18:00 (06:00 PM):

Timezone Date/Time
US / Pacific Tue 21-Dec-2021 10:00 (10:00 AM)
US / Mountain Tue 21-Dec-2021 11:00 (11:00 AM)
US / Central Tue 21-Dec-2021 12:00 (12:00 PM)
US / Eastern Tue 21-Dec-2021 13:00 (01:00 PM)
EU / Western Tue 21-Dec-2021 18:00 (06:00 PM)
EU / Central Tue 21-Dec-2021 19:00 (07:00 PM)
EU / Eastern Tue 21-Dec-2021 20:00 (08:00 PM)
Moscow Tue 21-Dec-2021 21:00 (09:00 PM)
Chennai Tue 21-Dec-2021 23:30 (11:30 PM)
Hangzhou Wed 22-Dec-2021 02:00 (02:00 AM)
Tokyo Wed 22-Dec-2021 03:00 (03:00 AM)
Sydney Wed 22-Dec-2021 05:00 (05:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2021-09-28

Time

UTC Tue 28-Sep-2021 17:00 (05:00 PM):

Timezone Date/Time
US / Pacific Tue 28-Sep-2021 10:00 (10:00 AM)
US / Mountain Tue 28-Sep-2021 11:00 (11:00 AM)
US / Central Tue 28-Sep-2021 12:00 (12:00 PM)
US / Eastern Tue 28-Sep-2021 13:00 (01:00 PM)
EU / Western Tue 28-Sep-2021 18:00 (06:00 PM)
EU / Central Tue 28-Sep-2021 19:00 (07:00 PM)
EU / Eastern Tue 28-Sep-2021 20:00 (08:00 PM)
Moscow Tue 28-Sep-2021 20:00 (08:00 PM)
Chennai Tue 28-Sep-2021 22:30 (10:30 PM)
Hangzhou Wed 29-Sep-2021 01:00 (01:00 AM)
Tokyo Wed 29-Sep-2021 02:00 (02:00 AM)
Sydney Wed 29-Sep-2021 03:00 (03:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/node

  • module: ESM loader approach #36954

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2022-05-10

Time

UTC Tue 10-May-2022 18:00 (06:00 PM):

Timezone Date/Time
US / Pacific Tue 10-May-2022 11:00 (11:00 AM)
US / Mountain Tue 10-May-2022 12:00 (12:00 PM)
US / Central Tue 10-May-2022 13:00 (01:00 PM)
US / Eastern Tue 10-May-2022 14:00 (02:00 PM)
EU / Western Tue 10-May-2022 19:00 (07:00 PM)
EU / Central Tue 10-May-2022 20:00 (08:00 PM)
EU / Eastern Tue 10-May-2022 21:00 (09:00 PM)
Moscow Tue 10-May-2022 21:00 (09:00 PM)
Chennai Tue 10-May-2022 23:30 (11:30 PM)
Hangzhou Wed 11-May-2022 02:00 (02:00 AM)
Tokyo Wed 11-May-2022 03:00 (03:00 AM)
Sydney Wed 11-May-2022 04:00 (04:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2021-08-06

Time

UTC Fri 06-Aug-2021 17:00 (05:00 PM):

Timezone Date/Time
US / Pacific Fri 06-Aug-2021 10:00 (10:00 AM)
US / Mountain Fri 06-Aug-2021 11:00 (11:00 AM)
US / Central Fri 06-Aug-2021 12:00 (12:00 PM)
US / Eastern Fri 06-Aug-2021 13:00 (01:00 PM)
EU / Western Fri 06-Aug-2021 18:00 (06:00 PM)
EU / Central Fri 06-Aug-2021 19:00 (07:00 PM)
EU / Eastern Fri 06-Aug-2021 20:00 (08:00 PM)
Moscow Fri 06-Aug-2021 20:00 (08:00 PM)
Chennai Fri 06-Aug-2021 22:30 (10:30 PM)
Hangzhou Sat 07-Aug-2021 01:00 (01:00 AM)
Tokyo Sat 07-Aug-2021 02:00 (02:00 AM)
Sydney Sat 07-Aug-2021 03:00 (03:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/node

  • esm: consolidate ESM Loader methods #37468

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2022-04-12

Time

UTC Tue 12-Apr-2022 18:00 (06:00 PM):

Timezone Date/Time
US / Pacific Tue 12-Apr-2022 11:00 (11:00 AM)
US / Mountain Tue 12-Apr-2022 12:00 (12:00 PM)
US / Central Tue 12-Apr-2022 13:00 (01:00 PM)
US / Eastern Tue 12-Apr-2022 14:00 (02:00 PM)
EU / Western Tue 12-Apr-2022 19:00 (07:00 PM)
EU / Central Tue 12-Apr-2022 20:00 (08:00 PM)
EU / Eastern Tue 12-Apr-2022 21:00 (09:00 PM)
Moscow Tue 12-Apr-2022 21:00 (09:00 PM)
Chennai Tue 12-Apr-2022 23:30 (11:30 PM)
Hangzhou Wed 13-Apr-2022 02:00 (02:00 AM)
Tokyo Wed 13-Apr-2022 03:00 (03:00 AM)
Sydney Wed 13-Apr-2022 04:00 (04:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2022-03-01

Time

UTC Tue 01-Mar-2022 19:00 (07:00 PM):

Timezone Date/Time
US / Pacific Tue 01-Mar-2022 11:00 (11:00 AM)
US / Mountain Tue 01-Mar-2022 12:00 (12:00 PM)
US / Central Tue 01-Mar-2022 13:00 (01:00 PM)
US / Eastern Tue 01-Mar-2022 14:00 (02:00 PM)
EU / Western Tue 01-Mar-2022 19:00 (07:00 PM)
EU / Central Tue 01-Mar-2022 20:00 (08:00 PM)
EU / Eastern Tue 01-Mar-2022 21:00 (09:00 PM)
Moscow Tue 01-Mar-2022 22:00 (10:00 PM)
Chennai Wed 02-Mar-2022 00:30 (12:30 AM)
Hangzhou Wed 02-Mar-2022 03:00 (03:00 AM)
Tokyo Wed 02-Mar-2022 04:00 (04:00 AM)
Sydney Wed 02-Mar-2022 06:00 (06:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/loaders

  • Add ambient-loader proposal #63

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2022-03-29

Time

UTC Tue 29-Mar-2022 18:00 (06:00 PM):

Timezone Date/Time
US / Pacific Tue 29-Mar-2022 11:00 (11:00 AM)
US / Mountain Tue 29-Mar-2022 12:00 (12:00 PM)
US / Central Tue 29-Mar-2022 13:00 (01:00 PM)
US / Eastern Tue 29-Mar-2022 14:00 (02:00 PM)
EU / Western Tue 29-Mar-2022 19:00 (07:00 PM)
EU / Central Tue 29-Mar-2022 20:00 (08:00 PM)
EU / Eastern Tue 29-Mar-2022 21:00 (09:00 PM)
Moscow Tue 29-Mar-2022 21:00 (09:00 PM)
Chennai Tue 29-Mar-2022 23:30 (11:30 PM)
Hangzhou Wed 30-Mar-2022 02:00 (02:00 AM)
Tokyo Wed 30-Mar-2022 03:00 (03:00 AM)
Sydney Wed 30-Mar-2022 05:00 (05:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

List of loaders in the wild today

The goal of this issue is to track a list of loaders used in the wild today. Eventually, we will need to decide how loaders compose and interop. It will be helpful to keep track of what loaders are doing today so we understand how to support their use-cases.

This list is very short to start, but I'll do my best to ask around in forums and mention this issue to the authors.

Yarn2 PnP

ts-node

  • First released in v8.10.0 as an experimental feature to match node's experimental status
  • Documentation, feedback, Q&A in this thread: TypeStrong/ts-node#1007

@istanbuljs/esm-loader-hook

  • NPM
  • I don't know if this is commonly used. I need to ask the authors

Jest

multiloader

quibble

babel-register-esm

Node.js Loaders Team Meeting 2021-06-25

Time

UTC Fri 25-Jun-2021 17:00 (05:00 PM):

Timezone Date/Time
US / Pacific Fri 25-Jun-2021 10:00 (10:00 AM)
US / Mountain Fri 25-Jun-2021 11:00 (11:00 AM)
US / Central Fri 25-Jun-2021 12:00 (12:00 PM)
US / Eastern Fri 25-Jun-2021 13:00 (01:00 PM)
EU / Western Fri 25-Jun-2021 18:00 (06:00 PM)
EU / Central Fri 25-Jun-2021 19:00 (07:00 PM)
EU / Eastern Fri 25-Jun-2021 20:00 (08:00 PM)
Moscow Fri 25-Jun-2021 20:00 (08:00 PM)
Chennai Fri 25-Jun-2021 22:30 (10:30 PM)
Hangzhou Sat 26-Jun-2021 01:00 (01:00 AM)
Tokyo Sat 26-Jun-2021 02:00 (02:00 AM)
Sydney Sat 26-Jun-2021 03:00 (03:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/node

  • esm: consolidate ESM Loader methods #37468
  • module: ESM loader approach #36954

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2021-08-20

Time

UTC Fri 20-Aug-2021 17:00 (05:00 PM):

Timezone Date/Time
US / Pacific Fri 20-Aug-2021 10:00 (10:00 AM)
US / Mountain Fri 20-Aug-2021 11:00 (11:00 AM)
US / Central Fri 20-Aug-2021 12:00 (12:00 PM)
US / Eastern Fri 20-Aug-2021 13:00 (01:00 PM)
EU / Western Fri 20-Aug-2021 18:00 (06:00 PM)
EU / Central Fri 20-Aug-2021 19:00 (07:00 PM)
EU / Eastern Fri 20-Aug-2021 20:00 (08:00 PM)
Moscow Fri 20-Aug-2021 20:00 (08:00 PM)
Chennai Fri 20-Aug-2021 22:30 (10:30 PM)
Hangzhou Sat 21-Aug-2021 01:00 (01:00 AM)
Tokyo Sat 21-Aug-2021 02:00 (02:00 AM)
Sydney Sat 21-Aug-2021 03:00 (03:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/node

  • esm: consolidate ESM Loader methods #37468

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

How will hot reloading be accomplished?

I see in the use cases documentation that it mentions hot reloading as a use case that you're hoping to support - is there a proposal for how that would be accomplished?

I am interested in it for the various node loaders that I help maintain at https://github.com/node-loader, such as an import maps loader, babel loader, http loader, postcss loader, etc. I've been using a user-land implementation of loader chaining to combine these, but am interested in also hot reloading them. I explored it a bit with adding query parameters to the module urls during resolve(), which succeeds in reinstantiating the module but does not delete or replace the older, existing module.

I also help maintain SystemJS, which has its own loader implementation, so I'm interested to see if the nodejs implementation has any similarities.

Node.js Loaders Team Meeting 2022-01-11

Time

UTC Tue 11-Jan-2022 19:00 (07:00 PM):

Timezone Date/Time
US / Pacific Tue 11-Jan-2022 11:00 (11:00 AM)
US / Mountain Tue 11-Jan-2022 12:00 (12:00 PM)
US / Central Tue 11-Jan-2022 13:00 (01:00 PM)
US / Eastern Tue 11-Jan-2022 14:00 (02:00 PM)
EU / Western Tue 11-Jan-2022 19:00 (07:00 PM)
EU / Central Tue 11-Jan-2022 20:00 (08:00 PM)
EU / Eastern Tue 11-Jan-2022 21:00 (09:00 PM)
Moscow Tue 11-Jan-2022 22:00 (10:00 PM)
Chennai Wed 12-Jan-2022 00:30 (12:30 AM)
Hangzhou Wed 12-Jan-2022 03:00 (03:00 AM)
Tokyo Wed 12-Jan-2022 04:00 (04:00 AM)
Sydney Wed 12-Jan-2022 06:00 (06:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/node

  • esm: implement the getFileSystem hook #41076

nodejs/loaders

  • Loader chaining while loading loaders #56

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2021-09-21

Time

UTC Tue 21-Sep-2021 17:00 (05:00 PM):

Timezone Date/Time
US / Pacific Tue 21-Sep-2021 10:00 (10:00 AM)
US / Mountain Tue 21-Sep-2021 11:00 (11:00 AM)
US / Central Tue 21-Sep-2021 12:00 (12:00 PM)
US / Eastern Tue 21-Sep-2021 13:00 (01:00 PM)
EU / Western Tue 21-Sep-2021 18:00 (06:00 PM)
EU / Central Tue 21-Sep-2021 19:00 (07:00 PM)
EU / Eastern Tue 21-Sep-2021 20:00 (08:00 PM)
Moscow Tue 21-Sep-2021 20:00 (08:00 PM)
Chennai Tue 21-Sep-2021 22:30 (10:30 PM)
Hangzhou Wed 22-Sep-2021 01:00 (01:00 AM)
Tokyo Wed 22-Sep-2021 02:00 (02:00 AM)
Sydney Wed 22-Sep-2021 03:00 (03:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/node

  • module: ESM loader approach #36954

nodejs/loaders

  • Real life analysis: Yarn's loader #28
  • Feature request: Fine grained API to implement loaders that can do what --experimental-specifier-resolution does and more #26

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2022-05-24

Time

UTC Tue 24-May-2022 18:00 (06:00 PM):

Timezone Date/Time
US / Pacific Tue 24-May-2022 11:00 (11:00 AM)
US / Mountain Tue 24-May-2022 12:00 (12:00 PM)
US / Central Tue 24-May-2022 13:00 (01:00 PM)
US / Eastern Tue 24-May-2022 14:00 (02:00 PM)
EU / Western Tue 24-May-2022 19:00 (07:00 PM)
EU / Central Tue 24-May-2022 20:00 (08:00 PM)
EU / Eastern Tue 24-May-2022 21:00 (09:00 PM)
Moscow Tue 24-May-2022 21:00 (09:00 PM)
Chennai Tue 24-May-2022 23:30 (11:30 PM)
Hangzhou Wed 25-May-2022 02:00 (02:00 AM)
Tokyo Wed 25-May-2022 03:00 (03:00 AM)
Sydney Wed 25-May-2022 04:00 (04:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Logging on errors thrown in loader

I am trying to debug a loader, but when it crashes I can't seem to log. Here is the trace I get.

I want to wrap these error messages and log tips for fixing them to the user. It doesn't seem possible.

node:internal/errors:465
    ErrorCaptureStackTrace(err);
    ^

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for /xxx/src/index.ts
    at new NodeError (node:internal/errors:372:5)
    at Object.getFileProtocolModuleFormat [as file:] (node:internal/modules/esm/get_format:80:11)
    at defaultGetFormat (node:internal/modules/esm/get_format:122:38)
    at defaultLoad (node:internal/modules/esm/load:21:20)
    at ESMLoader.load (node:internal/modules/esm/loader:431:26)
    at ESMLoader.moduleProvider (node:internal/modules/esm/loader:350:22)
    at new ModuleJob (node:internal/modules/esm/module_job:66:26)
    at #createModuleJob (node:internal/modules/esm/loader:369:17)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:328:34)
    at async ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:82:21) {
  code: 'ERR_UNKNOWN_FILE_EXTENSION'
}

Node.js Loaders Team Meeting 2021-10-26

Time

UTC Tue 26-Oct-2021 17:00 (05:00 PM):

Timezone Date/Time
US / Pacific Tue 26-Oct-2021 10:00 (10:00 AM)
US / Mountain Tue 26-Oct-2021 11:00 (11:00 AM)
US / Central Tue 26-Oct-2021 12:00 (12:00 PM)
US / Eastern Tue 26-Oct-2021 13:00 (01:00 PM)
EU / Western Tue 26-Oct-2021 18:00 (06:00 PM)
EU / Central Tue 26-Oct-2021 19:00 (07:00 PM)
EU / Eastern Tue 26-Oct-2021 20:00 (08:00 PM)
Moscow Tue 26-Oct-2021 20:00 (08:00 PM)
Chennai Tue 26-Oct-2021 22:30 (10:30 PM)
Hangzhou Wed 27-Oct-2021 01:00 (01:00 AM)
Tokyo Wed 27-Oct-2021 02:00 (02:00 AM)
Sydney Wed 27-Oct-2021 04:00 (04:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/loaders

  • update to support chaining & incorporate short-circuit as a return flag #41
  • Iterative chaining design #32

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Design Overview Documentation says 'code' but could be really any 'data'

https://github.com/nodejs/loaders/blob/main/doc/design/overview.md

2. load: Takes the resolved URL and returns runnable code (JavaScript, Wasm, etc.) as well as the name of one of Node’s ESM loader’s “translators”:

returns runnable code includes 'json' which isn't runnable code, it's just a single data object... Think the phrasing of that could be more inclusive.

Is only the 'json' internal translator able to just return an object? I see that this has a distinct enable flag --experimental-json-modules

Node.js Loaders Team Meeting 2021-09-15

Time

UTC Wed 15-Sep-2021 21:00 (09:00 PM):

2 pm PT / 5 pm ET / 9 pm UTC / 11 pm CEDT

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

Postponed until next meeting:

  • Real life analysis: Yarn’s loader #28
  • Feature request: Fine grained API to implement loaders that can do what –experimental-specifier-resolution does and more #26

Invited

  • Loaders team: @nodejs/loaders

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2022-02-01

Time

UTC Tue 01-Feb-2022 19:00 (07:00 PM):

Timezone Date/Time
US / Pacific Tue 01-Feb-2022 11:00 (11:00 AM)
US / Mountain Tue 01-Feb-2022 12:00 (12:00 PM)
US / Central Tue 01-Feb-2022 13:00 (01:00 PM)
US / Eastern Tue 01-Feb-2022 14:00 (02:00 PM)
EU / Western Tue 01-Feb-2022 19:00 (07:00 PM)
EU / Central Tue 01-Feb-2022 20:00 (08:00 PM)
EU / Eastern Tue 01-Feb-2022 21:00 (09:00 PM)
Moscow Tue 01-Feb-2022 22:00 (10:00 PM)
Chennai Wed 02-Feb-2022 00:30 (12:30 AM)
Hangzhou Wed 02-Feb-2022 03:00 (03:00 AM)
Tokyo Wed 02-Feb-2022 04:00 (04:00 AM)
Sydney Wed 02-Feb-2022 06:00 (06:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2021-07-23

Time

UTC Fri 23-Jul-2021 17:00 (05:00 PM):

Timezone Date/Time
US / Pacific Fri 23-Jul-2021 10:00 (10:00 AM)
US / Mountain Fri 23-Jul-2021 11:00 (11:00 AM)
US / Central Fri 23-Jul-2021 12:00 (12:00 PM)
US / Eastern Fri 23-Jul-2021 13:00 (01:00 PM)
EU / Western Fri 23-Jul-2021 18:00 (06:00 PM)
EU / Central Fri 23-Jul-2021 19:00 (07:00 PM)
EU / Eastern Fri 23-Jul-2021 20:00 (08:00 PM)
Moscow Fri 23-Jul-2021 20:00 (08:00 PM)
Chennai Fri 23-Jul-2021 22:30 (10:30 PM)
Hangzhou Sat 24-Jul-2021 01:00 (01:00 AM)
Tokyo Sat 24-Jul-2021 02:00 (02:00 AM)
Sydney Sat 24-Jul-2021 03:00 (03:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/node

  • esm: consolidate ESM Loader methods #37468
  • module: ESM loader approach #36954

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2021-05-14

Time

UTC Fri 14-May-2021 17:00 (05:00 PM):

Timezone Date/Time
US / Pacific Fri 14-May-2021 10:00 (10:00 AM)
US / Mountain Fri 14-May-2021 11:00 (11:00 AM)
US / Central Fri 14-May-2021 12:00 (12:00 PM)
US / Eastern Fri 14-May-2021 13:00 (01:00 PM)
EU / Western Fri 14-May-2021 18:00 (06:00 PM)
EU / Central Fri 14-May-2021 19:00 (07:00 PM)
EU / Eastern Fri 14-May-2021 20:00 (08:00 PM)
Moscow Fri 14-May-2021 20:00 (08:00 PM)
Chennai Fri 14-May-2021 22:30 (10:30 PM)
Hangzhou Sat 15-May-2021 01:00 (01:00 AM)
Tokyo Sat 15-May-2021 02:00 (02:00 AM)
Sydney Sat 15-May-2021 03:00 (03:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/node

  • esm: consolidate ESM Loader methods #37468
  • Allow main thread to be supplanted by another #38454

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

What will happen if the loader thread terminates due to an error?

When loaders are moved off the main thread, what will happen if the loader thread crashes / terminates? Could be due to an uncaught error, code erroneously calling process.exit(), or any other reason I didn't think of.

Supposing the main thread makes a blocking / synchronous request to the loader thread using postMessage and Atomics.wait. We need to guard against hanging the main thread indefinitely in the event the loader thread crashes. The motivation is running a compiler in the loader thread and also querying the same compiler within synchronous require() hooks on the main thread.

In past threading work, I've done this with a watchdog thread and a worker thread, where the watchdog checks for the worker to terminate and wakes up the main thread, preventing the main thread from hanging.

If loaders are spec-ed such that any termination of the loader thread is guaranteed to crash the node process, then this watchdog may not be needed. However, if "crash the node process" means waiting for the main thread to respond to a process.on('unhandledException') or something like that, then we've still got the problem of a hanging main thread.

Node.js Loaders Team Meeting 2022-03-15

Time

UTC Tue 15-Mar-2022 18:00 (06:00 PM):

Timezone Date/Time
US / Pacific Tue 15-Mar-2022 11:00 (11:00 AM)
US / Mountain Tue 15-Mar-2022 12:00 (12:00 PM)
US / Central Tue 15-Mar-2022 13:00 (01:00 PM)
US / Eastern Tue 15-Mar-2022 14:00 (02:00 PM)
EU / Western Tue 15-Mar-2022 18:00 (06:00 PM)
EU / Central Tue 15-Mar-2022 19:00 (07:00 PM)
EU / Eastern Tue 15-Mar-2022 20:00 (08:00 PM)
Moscow Tue 15-Mar-2022 21:00 (09:00 PM)
Chennai Tue 15-Mar-2022 23:30 (11:30 PM)
Hangzhou Wed 16-Mar-2022 02:00 (02:00 AM)
Tokyo Wed 16-Mar-2022 03:00 (03:00 AM)
Sydney Wed 16-Mar-2022 05:00 (05:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/loaders

  • Add ambient-loader proposal #63

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2021-12-07

Time

UTC Tue 07-Dec-2021 18:00 (06:00 PM):

Timezone Date/Time
US / Pacific Tue 07-Dec-2021 10:00 (10:00 AM)
US / Mountain Tue 07-Dec-2021 11:00 (11:00 AM)
US / Central Tue 07-Dec-2021 12:00 (12:00 PM)
US / Eastern Tue 07-Dec-2021 13:00 (01:00 PM)
EU / Western Tue 07-Dec-2021 18:00 (06:00 PM)
EU / Central Tue 07-Dec-2021 19:00 (07:00 PM)
EU / Eastern Tue 07-Dec-2021 20:00 (08:00 PM)
Moscow Tue 07-Dec-2021 21:00 (09:00 PM)
Chennai Tue 07-Dec-2021 23:30 (11:30 PM)
Hangzhou Wed 08-Dec-2021 02:00 (02:00 AM)
Tokyo Wed 08-Dec-2021 03:00 (03:00 AM)
Sydney Wed 08-Dec-2021 05:00 (05:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/loaders

  • Add hook for manifest loading #44
  • Design doc for reuse of defaultResolve #33
  • Feature request: Utility functions for the steps of resolution and loading #26

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Real life analysis: Yarn's loader

Cf #19 (comment)

I was in vacation at the time of the past meetings (and I needed some time to ramp up on our implementation, since I didn't write it myself), but I'd be available for the next one if you're still interested. Note that I can't add the loaders-agenda tag myself.

In the meantime, if you want to take an early peek, those are the two most relevant files from the implementation:

Node.js Loaders Team Meeting 2021-06-11

Time

UTC Fri 11-Jun-2021 17:00 (05:00 PM):

Timezone Date/Time
US / Pacific Fri 11-Jun-2021 10:00 (10:00 AM)
US / Mountain Fri 11-Jun-2021 11:00 (11:00 AM)
US / Central Fri 11-Jun-2021 12:00 (12:00 PM)
US / Eastern Fri 11-Jun-2021 13:00 (01:00 PM)
EU / Western Fri 11-Jun-2021 18:00 (06:00 PM)
EU / Central Fri 11-Jun-2021 19:00 (07:00 PM)
EU / Eastern Fri 11-Jun-2021 20:00 (08:00 PM)
Moscow Fri 11-Jun-2021 20:00 (08:00 PM)
Chennai Fri 11-Jun-2021 22:30 (10:30 PM)
Hangzhou Sat 12-Jun-2021 01:00 (01:00 AM)
Tokyo Sat 12-Jun-2021 02:00 (02:00 AM)
Sydney Sat 12-Jun-2021 03:00 (03:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/node

  • esm: consolidate ESM Loader methods #37468
  • module: ESM loader approach #36954

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Feature request: Utility functions for the steps of resolution and loading

Today it is possible to alter the resolve algorithm to some extent using --experimental-specifier-resolution. However that switch only supports a couple of fixed use-cases and is not available programmatically within a loader implementation. When implementing a custom loader it would be nice to have access to a fine grained API so we can re-use parts of the default resolve algorithm but still be able to interject whatever logic you want in between steps, or skip/replace steps, without having to re-implement the full algorithm. That way a custom loader can be composed from the API rather than have to re-implement the whole resolve algorithm.

The old node resolve algorithm was quite easy to re-implement so for that we did not really need a fine grained API. The new algorithm supports more features and is therefore more work to re-implement. So IMHO it would make sense to expose an API for the steps in this more complex algorithm rather than having custom loaders re-implement or copy it in full.

The use-cases for this would be for example implementing a loader for .ts files in ts-node where the algorithm is the same but only differ in file extension. Also, Yarn PnP could use this for its resolve. See below for those requests.


ORIGINAL POST (before the issue was re-formed as a feature request):

I'm sorry if this is not the right forum to ask this, if not pls let me know where it is appropriate to ask.

I'm curious about how the loader specification and --experimental-specifier-resolution interact. Specifically I'm looking into loading ES modules directly from typescript .ts files using any specifier (bare, relative). With the PR "esm: consolidate ESM Loader methods #37468" merged, this seems to have gotten easier. From what I understand it would now be possible to have something like this:

import { foo } from "./foo.ts"

And then just implement the loader hook as the resolve hook will not throw on the unknown .ts extension. This seems like a nice improvement as we don't have to re-implement the full resolve anymore. However most typescript projects are setup to import without extensions, like this:

import { foo } from "./foo"

I believe having this resolved is related to --experimental-specifier-resolution but AFAIK this only support .js. So in order to resolve the above to foo.ts the only way I see today is to re-implement the whole module resolution algorithm, which with the new spec is a rather big task. I think that is what ts-node is doing here, basically copying the impl from the internal nodejs loader.

I'm guessing the --experimental-specifier-resolution=node flag somehow augments the internal loader to check for .js and index.js (I have not looked deep into the source code though). The point being I think it would be nice if a loader could be allowed to somehow do the same augmentation. Specifically allowing imports without extension to be resolved to .ts files without having to re-implement the full resolution algorithm. Or is there some other way to achieve this?

Loaders and `exports` field

As a practical example, let's say that we have:

/node_modules/foo/package.json

{
  "name": "foo",
  "exports": {
    "./utils": "./sources/utils"
  }
}

/node_modules/foo/sources/utils.ts

// Content doesn't matter

/index.js

import 'foo/utils';

And let's say that we have a composed pipeline:

  • The PnP resolver

    • will get foo/utils as input
    • will want to turn the foo segment into /node_modules/foo (and leave the rest untouched)
  • The TS resolver

    • will want to add the.ts extension to whatever foo/utils resolves into

What should return the PnP resolver?

a. "/node_modules/foo/utils"
b. "/node_modules/foo/sources/utils"
c. "/node_modules/foo/sources/utils.js"
d. ["/node_modules/foo", "./utils"]

Option A is incorrect; the exports field doesn't apply when a file is loaded through an absolute path:

It is not a strong encapsulation since a direct require of any absolute subpath of the package such as require('/path/to/node_modules/pkg/subpath.js') will still load subpath.js.

Option B is good, but requires the PnP loader to not only resolve foo into node_modules/foo, but also ./util into ./sources/utils. To do that requires either to reimplement the exports resolution (with the risks in terms of correctness that come with it), or Node to provide a import.meta.resolveExports('foo/utils', '/node_modules/foo') utility that would do the job for us. This is what we currently do, by embedding a copy of resolve.exports into our loader.

Option C essentially prevents the TS loader from working, since the extension is already resolved.

Option D (which isn't supported by Node at the moment) lets the PnP loader resolves the bare identifier part which Node would presumably process through the same import.meta.resolveExports function described earlier, but keeping it internal.

Call for Contributors: Setting up imports & exports conditions definitions repo

Posting here as spillover from the discussion in nodejs/node#40708 where it's been justifiably blocked for Node.js to define the "deno" condition in its documentation.

As a result there is a need for a community resource that maintains the environment resolution conditions definitions in a way that can be maintained as a well-defined definition registry over time.

Effectively taking this section from the docs - https://nodejs.org/dist/latest-v17.x/docs/api/packages.html#conditions-definitions, and moving it into an external shared resource with proper process around how conditions can be defined and shared widely.

Once set up, we can then remove that section from the Node.js documentation and link to the shared community repo being maintained from the Node.js docs as the primary home of these definitions.

We need volunteers though to help set up process and maintenance around this effort. We could possibly arrange initial meetings etc as needed to get to a point where things are running smoothly.

Please comment here if you would like to contribute to this effort and assist in getting a process set up to ensure this can be well-maintained. I think we would need at least two other people to volunteer to assist to move this effort forward.

Will Service Workers be considered as a use case?

Service Workers are different.
They are registered by a script.
But restarting/reloading the app with a service worker in control will intercept requests.
This incudes requests for import
A Service Worker can receive messages via postMessage.
A Loader has a message port too.
A Loader can preload global code, but it is also possible to load source code and augment it.

So I definitely see how a loader could be used to create a similar experience to controlling service workers, even though the registration/installation process is a bit different to service workers.

Is that a use case that is or maybe should be considered by this group?
Or was it considered?

If loaders are here to stay and would in principle enable all features service workers enable, that would be amazing.

This would help to write almost no overhead frameworks or smal shims for service worker code which could run in browsers and nodejs with no changes, which would be awesome :-)

Tool config transpilation for webpack, gulp, etc

I'm trying to best understand what a user's and tool author's experiences will be in the future when:

a) users write a project in pure ESM with package.json "type": "module"
b) users write a config file for a tool -- webpack, gulp, etc -- in a transpiled language, for example TypeScript or CoffeeScript
c) tool author does not like asking users to prefix with NODE_OPTIONS='--loader xyz', use cross-env on windows, etc

I've read the meeting minutes and the issues on nodejs/node and I haven't seen this use-case discussed lately. Has this been considered? If not, I can describe the challenges in greater detail in this issue.

Proposal: preImport loader hook

A preImport hook can take the place of the async use cases for having an async resolver hook by allowing any async work to be done upfront before triggering the further pipeline steps. For example, loading an import map could be an async operation prior to synchronously using that import map in resolution (which is exactly what the browser does).

Ultimately, the goal would be that by covering these needs, this should allow for a synchronous core resolver which would enable unification with browser resolution.

The initial hook PR would not deprecate the resolver yet, and could be released to get feedback on this first, before following up with a subsequent async resolver deprecation.

There is a basic draft API in nodejs/node#43245, with the following documentation:

preImport(specifier, context)
* `specifier` {string}
* `context` {Object}
  * `conditions` {string\[]} Resolution conditions of the current environment,
    as defined for the `package.json` imports and exports fields
  * `dynamic` {boolean} Whether this import is a dynamic `import()`
  * `importAssertions` {Object}
  * `parentURL` {string|undefined} The module importing this one, or undefined
    if this is the Node.js entry point

The preImport hook allows for tracking and asynchronous setup work for every
top-level import operation.
The preImport hook is called for each top-level import operation by the
module loader, both for the host-called imports (ie for the main entry) and for
dynamic import() imports. These are distinguished by the dynamic context.
All preImport hooks for all loaders are run asynchronously in parallel,
and block any further load operations (ie resolve and load) for the module graph
being imported until they all complete successfully.
Multiple import calls to the same import specifier will re-call the hook
multiple times. The first error thrown by the preImport hooks will be directly
returned to the specific import operation as the load failure.

export async function preImport (specifier, context) {
  if (context.topLevel)
    console.log(`Top-level load of ${specifier}`);
  else
    console.log(`Dynamic import of ${specifier} in ${context.parentURL}`);
}

Node.js Loaders Team Meeting 2021-07-09

Time

UTC Fri 09-Jul-2021 17:00 (05:00 PM):

Timezone Date/Time
US / Pacific Fri 09-Jul-2021 10:00 (10:00 AM)
US / Mountain Fri 09-Jul-2021 11:00 (11:00 AM)
US / Central Fri 09-Jul-2021 12:00 (12:00 PM)
US / Eastern Fri 09-Jul-2021 13:00 (01:00 PM)
EU / Western Fri 09-Jul-2021 18:00 (06:00 PM)
EU / Central Fri 09-Jul-2021 19:00 (07:00 PM)
EU / Eastern Fri 09-Jul-2021 20:00 (08:00 PM)
Moscow Fri 09-Jul-2021 20:00 (08:00 PM)
Chennai Fri 09-Jul-2021 22:30 (10:30 PM)
Hangzhou Sat 10-Jul-2021 01:00 (01:00 AM)
Tokyo Sat 10-Jul-2021 02:00 (02:00 AM)
Sydney Sat 10-Jul-2021 03:00 (03:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/node

  • esm: consolidate ESM Loader methods #37468
  • module: ESM loader approach #36954

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Feature: Workerized JS Module Script Loading

The future (and hopefully very near future) way to achieve performative module script loading will likely come in the form of implementing “worklet infrastructure” in core since it does not seem like a userland solution should be reasonably expected.

Worklets are a piece of specification infrastructure which can be used for running scripts independent of the main JavaScript execution environment, while not requiring any particular implementation model.
https://html.spec.whatwg.org/multipage/worklets.html

This issue should help us track progress in this area, which I find to be of great interest.

Node.js Loaders Team Meeting 2021-05-07

Time

UTC Fri 07-May-2021 17:00 (05:00 PM):

Timezone Date/Time
US / Pacific Fri 07-May-2021 10:00 (10:00 AM)
US / Mountain Fri 07-May-2021 11:00 (11:00 AM)
US / Central Fri 07-May-2021 12:00 (12:00 PM)
US / Eastern Fri 07-May-2021 13:00 (01:00 PM)
EU / Western Fri 07-May-2021 18:00 (06:00 PM)
EU / Central Fri 07-May-2021 19:00 (07:00 PM)
EU / Eastern Fri 07-May-2021 20:00 (08:00 PM)
Moscow Fri 07-May-2021 20:00 (08:00 PM)
Chennai Fri 07-May-2021 22:30 (10:30 PM)
Hangzhou Sat 08-May-2021 01:00 (01:00 AM)
Tokyo Sat 08-May-2021 02:00 (02:00 AM)
Sydney Sat 08-May-2021 03:00 (03:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/node

  • module: ESM loaders next steps #36396

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2022-01-18

Time

UTC Tue 18-Jan-2022 19:00 (07:00 PM):

Timezone Date/Time
US / Pacific Tue 18-Jan-2022 11:00 (11:00 AM)
US / Mountain Tue 18-Jan-2022 12:00 (12:00 PM)
US / Central Tue 18-Jan-2022 13:00 (01:00 PM)
US / Eastern Tue 18-Jan-2022 14:00 (02:00 PM)
EU / Western Tue 18-Jan-2022 19:00 (07:00 PM)
EU / Central Tue 18-Jan-2022 20:00 (08:00 PM)
EU / Eastern Tue 18-Jan-2022 21:00 (09:00 PM)
Moscow Tue 18-Jan-2022 22:00 (10:00 PM)
Chennai Wed 19-Jan-2022 00:30 (12:30 AM)
Hangzhou Wed 19-Jan-2022 03:00 (03:00 AM)
Tokyo Wed 19-Jan-2022 04:00 (04:00 AM)
Sydney Wed 19-Jan-2022 06:00 (06:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/node

  • make experimental-specifier-resolution=node support extension-less files #41465
  • esm: implement the getFileSystem hook #41076

nodejs/loaders

  • Loader chaining while loading loaders #56

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Team vs Working Group

It was never discussed with me (a core member who intends to make significant contributions and maintain the Node.js loader implementation) whether or not this should be a working group or a team. As you can see in the calendar that I made, my intention was to make it an official working group, so I am unsure why this was made into a team.


EDIT: After looking at the repo a bit more, it has become clear to me that it is still very much a WIP.

I assume everything is still up for debate as the members of this team/group are still not listed…

Node.js Loaders Team Meeting 2022-01-04

Time

UTC Tue 04-Jan-2022 19:00 (07:00 PM):

Timezone Date/Time
US / Pacific Tue 04-Jan-2022 11:00 (11:00 AM)
US / Eastern Tue 04-Jan-2022 14:00 (02:00 PM)
EU / Western Tue 04-Jan-2022 19:00 (07:00 PM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/node

  • esm: implement the getFileSystem hook #41076

nodejs/loaders

  • Loader chaining while loading loaders #56

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2021-09-03

Time

UTC Fri 03-Sep-2021 17:00 (05:00 PM):

Timezone Date/Time
US / Pacific Fri 03-Sep-2021 10:00 (10:00 AM)
US / Mountain Fri 03-Sep-2021 11:00 (11:00 AM)
US / Central Fri 03-Sep-2021 12:00 (12:00 PM)
US / Eastern Fri 03-Sep-2021 13:00 (01:00 PM)
EU / Western Fri 03-Sep-2021 18:00 (06:00 PM)
EU / Central Fri 03-Sep-2021 19:00 (07:00 PM)
EU / Eastern Fri 03-Sep-2021 20:00 (08:00 PM)
Moscow Fri 03-Sep-2021 20:00 (08:00 PM)
Chennai Fri 03-Sep-2021 22:30 (10:30 PM)
Hangzhou Sat 04-Sep-2021 01:00 (01:00 AM)
Tokyo Sat 04-Sep-2021 02:00 (02:00 AM)
Sydney Sat 04-Sep-2021 03:00 (03:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/node

  • esm: consolidate ESM Loader methods #37468

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2021-10-12

Time

UTC Tue 12-Oct-2021 17:00 (05:00 PM):

Timezone Date/Time
US / Pacific Tue 12-Oct-2021 10:00 (10:00 AM)
US / Mountain Tue 12-Oct-2021 11:00 (11:00 AM)
US / Central Tue 12-Oct-2021 12:00 (12:00 PM)
US / Eastern Tue 12-Oct-2021 13:00 (01:00 PM)
EU / Western Tue 12-Oct-2021 18:00 (06:00 PM)
EU / Central Tue 12-Oct-2021 19:00 (07:00 PM)
EU / Eastern Tue 12-Oct-2021 20:00 (08:00 PM)
Moscow Tue 12-Oct-2021 20:00 (08:00 PM)
Chennai Tue 12-Oct-2021 22:30 (10:30 PM)
Hangzhou Wed 13-Oct-2021 01:00 (01:00 AM)
Tokyo Wed 13-Oct-2021 02:00 (02:00 AM)
Sydney Wed 13-Oct-2021 04:00 (04:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/node

  • module: unflag import assertions #40250

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Node.js Loaders Team Meeting 2021-05-28

Time

UTC Fri 28-May-2021 17:00 (05:00 PM):

Timezone Date/Time
US / Pacific Fri 28-May-2021 10:00 (10:00 AM)
US / Mountain Fri 28-May-2021 11:00 (11:00 AM)
US / Central Fri 28-May-2021 12:00 (12:00 PM)
US / Eastern Fri 28-May-2021 13:00 (01:00 PM)
EU / Western Fri 28-May-2021 18:00 (06:00 PM)
EU / Central Fri 28-May-2021 19:00 (07:00 PM)
EU / Eastern Fri 28-May-2021 20:00 (08:00 PM)
Moscow Fri 28-May-2021 20:00 (08:00 PM)
Chennai Fri 28-May-2021 22:30 (10:30 PM)
Hangzhou Sat 29-May-2021 01:00 (01:00 AM)
Tokyo Sat 29-May-2021 02:00 (02:00 AM)
Sydney Sat 29-May-2021 03:00 (03:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/node

  • esm: consolidate ESM Loader methods #37468
  • module: ESM loader approach #36954

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Loader chaining *while loading loaders*

I didn't see this situation described in the doc: what's the plan for when loaders need to leverage the previous ones to setup the next ones? For example, take this command line:

node --loader pnp --loader ts-node foo.ts

The pnp loader is required to loader the dependencies from the right place. Not only this is true for the imports of foo.ts, but this is true for the ts-node loader itself, since it's a dependency as any other (Node wouldn't know where to find ts-node if the pnp loader wasn't available).

I think this is relatively easily solved, by stating that the ambient loader is augmented as loaders are loaded, not just once at the end, but I just wanted to check this had been formalized somewhere.

Node.js Loaders Team Meeting 2021-11-23

Time

UTC Tue 23-Nov-2021 18:00 (06:00 PM):

Timezone Date/Time
US / Pacific Tue 23-Nov-2021 10:00 (10:00 AM)
US / Mountain Tue 23-Nov-2021 11:00 (11:00 AM)
US / Central Tue 23-Nov-2021 12:00 (12:00 PM)
US / Eastern Tue 23-Nov-2021 13:00 (01:00 PM)
EU / Western Tue 23-Nov-2021 18:00 (06:00 PM)
EU / Central Tue 23-Nov-2021 19:00 (07:00 PM)
EU / Eastern Tue 23-Nov-2021 20:00 (08:00 PM)
Moscow Tue 23-Nov-2021 21:00 (09:00 PM)
Chennai Tue 23-Nov-2021 23:30 (11:30 PM)
Hangzhou Wed 24-Nov-2021 02:00 (02:00 AM)
Tokyo Wed 24-Nov-2021 03:00 (03:00 AM)
Sydney Wed 24-Nov-2021 05:00 (05:00 AM)

Or in your local time:

Links

Agenda

Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.

nodejs/loaders

  • Add hook for manifest loading #44
  • Design doc for reuse of defaultResolve #33
  • Feature request: Utility functions for the steps of resolution and loading #26

Invited

  • Loaders team: @nodejs/loaders

Observers/Guests

Notes

The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.

Joining the meeting

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.