#124 lets a kit express that it should create an external bundle. It also includes the external bundle's API and how that maps to a set of packages.

This task is about assembling the external bundle from those packages, and exporting them using the API defined in the kit config.

This should happen automatically as part of the @rnx-kit/cli command 'rnx-bundle'.

Use external bundle tech in internal repos and other OSS repos like FURN.

Add depcheck to the buildci lage pipeline (/package.json) and possible other pipelines. It is currently excluded because some work is needed to make it pass.

• As a first step, we should integrate rnx-align-deps in test app and have it consume some random capabilities.
• Investigate how to best verify that the packages defined in our profiles work.

Options like --reset-cache should be supported and passed through to the underlying CLI. Look at params for both bundle and start commands.

Full trace:

error: Error: Unable to parse name of '/~/node_modules/nanoid/non-secure'
    at resolveModule (/~/node_modules/@rnx-kit/metro-plugin-duplicates-checker/lib/gatherModules.js:40:15)
    at /~/node_modules/@rnx-kit/metro-plugin-duplicates-checker/lib/gatherModules.js:54:53
    at Array.forEach (<anonymous>)
    at gatherModulesFromSources (/~/node_modules/@rnx-kit/metro-plugin-duplicates-checker/lib/gatherModules.js:51:13)
    at Object.gatherModulesFromGraph (/~/node_modules/@rnx-kit/metro-plugin-duplicates-checker/lib/gatherModules.js:69:12)
    at Object.checkForDuplicateDependencies (/~/node_modules/@rnx-kit/metro-plugin-duplicates-checker/lib/checkForDuplicatePackages.js:48:52)
    at /~/node_modules/@rnx-kit/metro-plugin-duplicates-checker/lib/index.js:33:51
    at /~/node_modules/@rnx-kit/metro-serializer/lib/index.js:42:37
    at Array.forEach (<anonymous>)
    at /~/node_modules/@rnx-kit/metro-serializer/lib/index.js:42:17

This is due to an unconventional file structure of 'nanoid':

% find nanoid 
nanoid
nanoid/LICENSE
nanoid/bin
nanoid/bin/nanoid.cjs
nanoid/CHANGELOG.md
nanoid/index.browser.js
nanoid/async
nanoid/async/index.browser.js
nanoid/async/index.js
nanoid/async/package.json
nanoid/async/index.cjs
nanoid/async/index.native.js
nanoid/async/index.d.ts
nanoid/index.js
nanoid/README.md
nanoid/non-secure
nanoid/non-secure/index.js
nanoid/non-secure/package.json
nanoid/non-secure/index.cjs
nanoid/non-secure/index.d.ts
nanoid/package.json
nanoid/index.cjs
nanoid/url-alphabet
nanoid/url-alphabet/index.js
nanoid/url-alphabet/package.json
nanoid/url-alphabet/index.cjs
nanoid/nanoid.js
nanoid/index.d.ts

rnx-kit-scripts doesn't forward parameters to the underlying command. For instance, jest --watch is very useful for TDD but the extra argument is currently dropped:

% yarn test --watch
yarn run v1.22.10
$ rnx-kit-scripts test --watch
[8:36:06 AM] ■ started 'test'
[8:36:07 AM] ■ started 'jest:default'
[8:36:07 AM] ■ Running Jest
[8:36:07 AM] ■ /usr/local/Cellar/node@12/12.20.1/bin/node "/~/rnx-kit/node_modules/jest/bin/jest.js" --passWithNoTests --colors --runInBand

Another useful parameter is --coverage:

% yarn jest --coverage
yarn run v1.22.10
$ /~/rnx-kit/node_modules/.bin/jest --coverage
 PASS  test/index.test.js
  @rnx-kit/metro-config
    ✓ defaultWatchFolders() returns an empty list outside a monorepo (7 ms)
    ✓ defaultWatchFolders() returns packages in a monorepo (39 ms)
    ✓ excludeExtraCopiesOf() throws if a package is not found (3 ms)
    ✓ exclusionList() ignores extra copies of react and react-native (10 ms)
    ✓ exclusionList() returns additional exclusions (2 ms)
    ✓ makeBabelConfig() returns default Babel config
    ✓ makeBabelConfig() returns a Babel config with additional plugins (1 ms)
    ✓ makeMetroConfig() returns a default Metro config (289 ms)
    ✓ makeMetroConfig() merges Metro configs (3 ms)
    ✓ makeMetroConfig() merges `extraNodeModules` (2 ms)
    ✓ packs only necessary files (419 ms)

----------|---------|----------|---------|---------|-------------------
File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
----------|---------|----------|---------|---------|-------------------
All files |     100 |    91.67 |     100 |     100 |                   
 index.js |     100 |    91.67 |     100 |     100 | 48,243            
----------|---------|----------|---------|---------|-------------------
Test Suites: 1 passed, 1 total
Tests:       11 passed, 11 total
Snapshots:   0 total
Time:        1.524 s, estimated 2 s
Ran all test suites.
✨  Done in 1.99s.

test-app-mobile should be expanded to work on macos and windows, and renamed to reflect this expansion.

If we need detailed bundle analysis for Metro, we're going to have to piece together the tooling ourselves.

Webpack has a plugin for bundle analysis: https://github.com/webpack-contrib/webpack-bundle-analyzer. This looks nice and is familiar to the community. Replicating this in Metro would be ideal.

Alternatively, there is a library which does bundle analysis using the source-map file: https://github.com/danvk/source-map-explorer. This could be used as a starting point, or as an alternative "feed" to power the webpack analyzer.

In rnx-kit packages, this babel config seems to be coming up in several places:

module.exports = {
  presets: [
    ["@babel/preset-env", { targets: { node: "current" } }],
    "@babel/preset-typescript",
  ],
};

This allows Jest to consume .ts files directly.

We should extract this, and any other common config, into the the rnx-kit-scripts package and share the config.

When enabling experimentalImportSupport in metro.config.js:

module.exports = {
  transformer: {
    getTransformOptions: async () => ({
      transform: {
        experimentalImportSupport: true,
        inlineRequires: true,
      },
    }),
  },
};

disableImportExportTransform should also be enabled in babel.config.js:

module.exports = {
  presets: [
    [
      "module:metro-react-native-babel-preset",
      { disableImportExportTransform: true },
    ],
  ],
};

When babel.config.js is missing, disableImportExportTransform is set to the value of experimentalImportSupport in metro-react-native-babel-transformer. Since we're recommending folks to use our preset, we need to find a way to keep both in sync somehow.

Implement a tool for making sure that Babel and Metro are correctly configured with respect to the repo setup, whether packages are vanilla or use external bundles, etc.

Haul/Webpack uses Circular Dependency Plugin.

Involves the following:

• Integrate semantic release into the repo
• Hook up automated tasks to publish. Likely publishing on change right now in the early phases
• Set up access tokens to allow the repo to publish to @rnx-kit scope on npmjs.org

Once TS validation is part of bundling [#113], bring it into FURN by updating the @rnx-kit/cli version. Verify that it runs during FURN builds, and that it doesn't slow things down more than expected. Also verify that it catches TS errors -- may need to introduce one to see it fail.

There are some examples of how to embed skipping into the CI YAML. Need to look into this more as we are currently double-building.

The repo should have a functional CI loop. For starters it should simply install and build as those are the only two things that work.

Long term I would propose that we have a buildci or runci script command that will go through the various pipelines as we add them. This allows the primary tasks of the CI to be run locally.

There are open compliance tasks that need to be reviewed for your rnx-kit repo.

Action required: 3 compliance tasks

To bring this repository to the standard required for 2021, we require administrators of this and all Microsoft GitHub repositories to complete a small set of tasks within the next 60 days. This is critical work to ensure the compliance and security of your microsoft GitHub organization.

Please take a few minutes to complete the tasks at: https://repos.opensource.microsoft.com/orgs/microsoft/repos/rnx-kit/compliance

• No Service Tree mapping has been set for this repo. If this team does not use Service Tree, they can also opt-out of providing Service Tree data in the Compliance tab.
• No repository maintainers are set. The Open Source Maintainers are the decision-makers and actionable owners of the repository, irrespective of administrator permission grants on GitHub.
• Classification of the repository as production/non-production is missing in the Compliance tab.

You can close this work item once you have completed the compliance tasks, or it will automatically close within a day of taking action.

If you no longer need this repository, it might be quickest to delete the repo, too.

GitHub inside Microsoft program information

More information about GitHub inside Microsoft and the new GitHub AE product can be found at https://aka.ms/gim or by contacting [email protected]

FYI: current admins at Microsoft include @JasonVMo, @tom-un, @tido64, @acoates-ms, @danrydholm, @kelset, @mielke, @afoxman

If a package has a different version of react-native installed locally, metro-config should ignore the one that is hoisted in repo/workspace root.

workspace
├── node_modules
│   └── [email protected]  <-- should be ignored
└── packages
    └── my-awesome-package
        └── node_modules
            └── [email protected]  <-- should take precedence

cc @tom-un @kelset

From CI run #114:

2021-04-03T18:46:00.5145612Z info ▶️ start @rnx-kit/test-app bundle
2021-04-03T18:46:00.5147278Z verb |  hash: 3b3f29beff9c5eea189f66bee2d17dbae7c09a74, cache hit? false
2021-04-03T18:46:00.5148899Z verb |  Running /tmp/yarn--1617475553959-0.7057094631993919/yarn run bundle
2021-04-03T18:46:00.5150065Z verb |  $ react-native rnx-bundle
2021-04-03T18:46:00.5151083Z verb |  Generating metro bundle(s)...
2021-04-03T18:46:00.5154325Z verb |  $ /home/runner/work/rnx-kit/rnx-kit/node_modules/.bin/react-native bundle --platform ios --entry-file ./lib/src/index.js --bundle-output /home/runner/work/rnx-kit/rnx-kit/packages/test-app/dist/main.ios.jsbundle --assets-dest ./dist --dev true --sourcemap-output /home/runner/work/rnx-kit/rnx-kit/packages/test-app/dist/main.ios.jsbundle.map
2021-04-03T18:46:00.5156577Z verb |                   Welcome to React Native!
2021-04-03T18:46:00.5157429Z verb |                  Learn once, write anywhere
2021-04-03T18:46:00.5159816Z verb |  error The resource `/home/runner/work/rnx-kit/rnx-kit/packages/test-app/src/lib/src/index.js` was not found. Run CLI with --verbose flag for more details.
2021-04-03T18:46:00.5161491Z verb |  Error: The resource `/home/runner/work/rnx-kit/rnx-kit/packages/test-app/src/lib/src/index.js` was not found.
2021-04-03T18:46:00.5162975Z verb |      at /home/runner/work/rnx-kit/rnx-kit/node_modules/metro/src/IncrementalBundler.js:157:26
2021-04-03T18:46:00.5163874Z verb |      at gotStat (fs.js:1864:21)
2021-04-03T18:46:00.5164605Z verb |      at FSReqCallback.oncomplete (fs.js:183:21)
2021-04-03T18:46:00.5165340Z verb |  error Command failed with exit code 1.
2021-04-03T18:46:00.5166300Z verb |  info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

cc @afoxman

While deduping workspace-tools, I saw that Beachball has released 2.x. I don't know if there are any breaking changes but we should bump to keep getting updates.

https://github.com/microsoft/boll lints packages in a monorepo, making sure they have all declared all dependencies correctly among other things.

cc @afoxman

We need to leverage dependabot to keep our external dependencies updated. I recently spent time investigating and diagnosing a beachball publishing issue, only to find out it had already been fixed. It would be far less time consuming to periodically review version bump PRs.

As we did for Metro's serializer, we need a plugin framework for Metro's resolver.

This will be used for plugins that handle symlinks and inject external bundle stubs.

May need to remove windows as a platform. Not sure what 'react-native bundle' supports. Needs some investigation.

Mapping from 'targets' to platform may be impacted as well.

Tree-shaking is the practice of finding and excluding unused JS code when assembling a bundle file. It has been historically absent from Metro, despite several attempts to get it added over the years. A prototype implementation has recently gained traction, and should be the starting point for this work.

Implement a tool for helping with RN upgrades, e.g. migrating a package on 0.62 to 0.63. It should be able to bump dependencies, remove packages that are deprecated, and make any necessary reconfigurations.

This tool should make use of the ones created in #139, #157, and #160.

Besides locking down the number of available options, I'm not quite sure what value rnx-start is supposed to bring. react-native start is already a very simple command for most scenarios.

Following the introduction of metro-serializer with plugin support in #115, we should make metro-plugin-duplicates-checker support it as well.

There are a number of ways source maps can "go wrong". They can get very large. File references can be unusable due to incorrect relative paths. Debuggers may get confused and refer to transpiled JS files when TS source files are available.

This issue tracks to work of making sure the source maps we produce with our metro configuration are portable and correct and as small as possible. If any issues are found, they should be logged as new bugs.

This package currently doesn't bundle because it is using yarn workspaces, we should get this working and make sure it is validated in CI.

Create an ES Lint configuration to use during builds and in IDEs like VSCode.

Verify that 'yarn build', 'yarn buildci', and VSCode all report the same lint errors, and are in fact synchronized to use the same ES Lint config.

ERROR: Attempting to bump to a version that already exists in the registry: @rnx-kit/[email protected]
Something went wrong with the publish! Manually update these package and versions:
- @rnx-kit/[email protected]
- @rnx-kit/[email protected]
No packages have been published
error Command failed with exit code 1.
Validating package version - @rnx-kit/[email protected] Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

See build logs: https://github.com/microsoft/rnx-kit/runs/2011746852

cc @afoxman

devs shouldn't have to go to the root and run "yarn build-tools" or run a full build of the repo when first cloning. the experience should be "yarn" to install deps, then go into a package a build it.

lage pipelines do the staging of build-tools before build/test/lint/etc... But those only run at the root.

One suggestion was to run build-tools only during yarn post-install.

I'd like to delay it further, if possible, and run it when build happens (e.g. part of the just config). That would mean always checking if build-tools were done, though, and we don't have access to lage's cache from there. Invoking lage is pretty slow so that's a non-starter for this approach.

Migrate these two packages to use the repo build/test scripts, rather their own custom commands in package.json.

Once #149 lands and is published, we can integrate it in 1JS.

• Integrate dep-check in a few packages only, and touching only a few dependencies. [ADO-802934]
  • We should let this simmer for a week to weed out any issues.
• Integrate dep-check in the rest of the packages. [ADO-818823]
• Allow dep-check to manage more dependencies.
  • ADO-831571
  • ADO-833130
  • ADO-833310
  • ADO-833337
  • ADO-833279
  • ADO-844952

Finalize on the npm scope and package names. Our preference is @rnx-kit as a scope. 1ES is strongly suggesting we use @microsoft.

Once that is finalized, we'll need to update package names and generate a new NPM publishing token. Store the publishing token under the NPM_TOKEN secret.

The current publishing token is mapped to my NPM account which is only temporary.

See discussion: #115 (review)

Our build system runs Jest once under the platform "default" and then under each specific platform "ios", "android", "windows", macos", "win32". The platform parameter selects a jest config, such as jest.config.ios.js. Jest doesn't actually know/care about platforms.

The issue is that our build system falls back jest.config.js (or package.json) when there's no platform-specific jest config. Which means it is duplicating what the "default" jest run is doing.

Instead, it should do nothing, and not even invoke jest.

An external bundle exports a collection of packages, each under a unique name. The external bundle author specifies this mapping as kit configuration.

For example, it might look like this:

"rnx-kit": {
    "externalBundle": {
        "interface": {
            "MyKitGlobal.Telemetry": "telemetryjs",
            "MyKitGlobal.StringResources": "string-resources",
            "MyKitGlobal.ReactNative.Components": "react-native"
        }
    }
}

The kit bundler will take this configuration, and use it to generate a programmatically-consumable, typesafe API. Something along the lines of a typescript .d.ts file.

Other packages which depend on this kit will naturally gain access to the typesafe API, and benefit from it in IDEs like VSCode and at lint/build time.

Haul currently produces smaller bundles than Metro. We need to investigate how we can narrow the gap.

Leads to explore:

• Haul's Babel preset is smaller than Metro's
• Tree shaking (see #127)

Issues:

• Allow options to be forwarded to metro-react-native-babel-preset (#136)
• Enabling experimentalImportSupport can help reduce the bundle size but must be done correctly (#142)

A tool should be able to run through dependencies and gather all requirements, then check whether the host satisfies them all. Apps can declare additional capabilities. If changes are needed, the tool should prompt what changes need to be made, and provide the option to make such changes automatically.

@react-native-community/cli has established a standard that command-line params are kebab-case rather than camelCase. The new CLI uses camelCase params, and should instead conform to the expected standard.

See discussion: #149 (comment)

Whenever #109 and #118 land.

This is faster and more robust than the yarn v1 that we are using. This approach should likely be based on the code in https://dev.azure.com/office/Office/_git/1JS?path=%2Fmidgard%2Fpackages%2Fyarn-fast which appears to be the most fully product ready version.

Some notes:

• This should likely be a standalone package that can be published and installed.
• It looks like there is room for some options in that implementation to cover use cases for other projects such as rnw.
• In midgard this also triggers a preinstall step "node tools/enforceYarnFast.js && node tools/check-yarn-lock.js"
• Users then install via the following script: "fast": "node packages/yarn-fast"

With regards to some of the options, react-native-windows runs it as:

npx [email protected] --network-concurrency 40 --frozen-lockfile --cache-folder \.yarnCache

It's worth following up on what prompted those settings and whether or not they should be defaults for everyone.

Building on the work of #133, and existing internal code, add a plugin which resolves symlinked files in Metro.