This looks like a cool project!

npx midnight-smoker test
npx: installed 59 in 3.207s

Cannot find module 'node:fs/promises'
Require stack:
-/.npm/_npx/31073/lib/node_modules/midnight-smoker/src/index.js
-/.npm/_npx/31073/lib/node_modules/midnight-smoker/src/cli.js

It'd be great if you supported node 14 LTS, as many CI environments for which this would be useful are still on older node versions.

Add a --config option. This would not make sense in a config file.

Each instance of SmokerError should contain a code prop with some constant string error code, e.g., EBOGART.

Tests depending on these should make assertions upon the error code instead of the message.

midnight-smoker should be able to run against multiple package managers. This is critical for projects which have some sort of dependency on the behavior of package managers.

Essentially, the endgame would be a "build matrix" for package managers. I'm a little surprised this sort of thing isn't already supported by something like GitHub Actions.

Well, off is, but not warn. If a RuleFailure has severity: 'warn', then do not process.exitCode = 1.

Given we want to use a single config file, and certain workspaces may need to override certain settings, we should figure that out. This would mimic what ESLint does.

Might get hairy w/ Zod due to the recursion.

It'd be helpful to be able to bring your own test framework, for example. The problem is that we cannot install any dev dependencies whatsoever, as it will skew the results due to node's module resolution algorithm. This is compounded by how some packages work, e.g., ava, which wants import {test} from 'ava' in each test file. Ideally, ava would not be able to be resolved from any script run by the smoke test, nor would any (transitive or no) dependency of ava. Using --global-style for installing these dev deps may work. The user may need to be explicit about which extra deps are required.

This is a thing that I find kind of frustrating about npm; npm run --workspaces <script> expects script to exist in all workspaces, and if it does not, it will fail. lerna's default behavior, OTOH, will ignore any workspace without the script.

There are tradeoffs here, certainly, but it'd be nice to have the option to mimic lerna's behavior.

This rule would check that known values in package.json are of the correct format. e.g., engines should be a Record<string,string>; not a string|string[].

I'm sure there's some tool that does this already, but it might be nice to do anyhow, since the use of such a tool does not seem popular.

Zod should be able to handle this.

Also the license field should be a valid SPDX id

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

• fix(deps): replace dependency read-pkg-up with read-package-up
• chore(deps): update actions/checkout action to v4.1.5
• chore(deps): update actions/setup-node action to v4.0.2
• chore(deps): update bahmutov/npm-install action to v1.8.40
• chore(deps): update definitelytyped (patch) (@types/is-file-esm, @types/node, @types/semver, @types/sinon)
• chore(deps): update dependency @tsconfig/node16 to v16.1.3
• chore(deps): update dependency prettier-plugin-pkg to v0.18.1
• chore(deps): update dependency typedoc to v0.25.13
• chore(deps): update dependency typedoc-plugin-zod to v1.1.2
• chore(deps): update dependency zod-to-json-schema to v3.22.5
• chore(deps): update google-github-actions/release-please-action action to v4.0.3
• chore(deps): update wagoid/commitlint-github-action action to v5.4.6
• fix(deps): update dependency glob to v10.3.15
• fix(deps): update dependency zod to v3.22.5
• chore(deps): update bahmutov/npm-install action to v1.10.2
• chore(deps): update dependency @types/node to v20.12.11
• chore(deps): update dependency markdownlint-cli2 to v0.13.0
• chore(deps): update dependency mocha to v10.4.0
• chore(deps): update dependency prettier to v3.2.5
• chore(deps): update dependency prettier-plugin-jsdoc to v1.3.0
• chore(deps): update dependency type-fest to v4.18.2
• chore(deps): update dependency zod-to-json-schema to v3.23.0
• chore(deps): update google-github-actions/release-please-action action to v4.1.0
• chore(deps): update wagoid/commitlint-github-action action to v5.5.1
• fix(deps): update dependency corepack to v0.28.1
• fix(deps): update dependency semver to v7.6.2
• fix(deps): update dependency zod to v3.23.8
• chore(deps): update actions/configure-pages action to v5
• chore(deps): update dependency dependency-cruiser to v16
• chore(deps): update dependency husky to v9
• chore(deps): update wagoid/commitlint-github-action action to v6
• fix(deps): update dependency lilconfig to v3
• fix(deps): update dependency read-pkg-up to v11
• fix(deps): update dependency zod-validation-error to v3
• 🔐 Create all rate-limited PRs at once 🔐

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): replace dependency npm-run-all with npm-run-all2 5.0.0
• chore(deps): update dependency typescript to v5.4.5
• chore(deps): update eslint-related packages (minor) (@typescript-eslint/eslint-plugin, @typescript-eslint/parser, eslint)
• chore(deps): update actions/deploy-pages action to v4
• chore(deps): update actions/setup-node action to v4
• chore(deps): update actions/upload-pages-artifact action to v3
• chore(deps): update commitlint monorepo to v19 (major) (@commitlint/cli, @commitlint/config-conventional)
• chore(deps): update dependency lint-staged to v15
• chore(deps): update dependency sinon to v17
• chore(deps): update eslint-related packages (major) (@typescript-eslint/eslint-plugin, @typescript-eslint/parser, eslint, eslint-config-prettier, eslint-plugin-n)
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

Currently we check main if the package is not a ESM package, and we check bin either way. We should check browser and types either way, as well.

It should ignore node_modules by default. Add a new option for this. The option should support globs.

See https://nodejs.org/docs/latest/api/cli.html#--frozen-intrinsics

This is a continuation of #249 and the work done in #273

This list may be very long. See if there's a reasonable way to display it.

given the ballooning featureset, I'm not sure if the README.md is really going to cut it anymore. need a docs site.

Let's make sure we aren't publishing a bunch of temp files, cache files, etc. These aren't "sensitive", but a nuisance nonetheless.

This is incorrect. It seems to me that midnight-smoker does not work with Node v14 because it requires npm@7 starting in v1.1.1...v1.2.1

The version of npm that ships with Node 14 is "6.14.18". Node 16 ships with npm "8.19.4".

For example: https://github.com/DavidAnson/markdownlint-cli2/actions/runs/6033354568

The same configuration passes on macOS and Ubuntu. See also #338.

Run npm exec --yes -- midnight-smoker --verbose test-invoke-as-cli
  npm exec --yes -- midnight-smoker --verbose test-invoke-as-cli
  shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
  
💨 midnight-smoker v7.0.1
- Packing current project…
✖ Failed while packing!

PackError: Package manager "npm" failed to pack
    at Npm7.pack (C:\npm\cache\_npx\b66bfc48521ee935\node_modules\midnight-smoker\src\pm\npm7.ts:180:13)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Smoker.pack (C:\npm\cache\_npx\b66bfc48521ee935\node_modules\midnight-smoker\src\smoker.ts:296:20)
    at Smoker.smoke (C:\npm\cache\_npx\b66bfc48521ee935\node_modules\midnight-smoker\src\smoker.ts:636:34)
    at Object.handler (C:\npm\cache\_npx\b66bfc48521ee935\node_modules\midnight-smoker\src\cli.ts:424:15) {
  code: 'ESMOKER_PACK',
  [cause]: {
    pm: 'npm',
    error: Error: Command failed with exit code 1: C:\hostedtoolcache\windows\node\20.5.1\x64\node.exe C:\npm\cache\_npx\b66bfc48521ee935\node_modules\.bin\corepack [email protected] pack --json --pack-destination=C:\Users\RUNNER~1\AppData\Local\Temp\midnight-smoker-W4Q28f --foreground-scripts=false
    C:\npm\cache\_npx\b66bfc48521ee935\node_modules\.bin\corepack:2
    basedir=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')")
              ^^^^^^^
    
    SyntaxError: missing ) after argument list
        at internalCompileFunction (node:internal/vm:73:18)
        at wrapSafe (node:internal/modules/cjs/loader:1153:20)
        at Module._compile (node:internal/modules/cjs/loader:1197:27)
        at Module._extensions..js (node:internal/modules/cjs/loader:1287:10)
        at Module.load (node:internal/modules/cjs/loader:1091:32)
        at Module._load (node:internal/modules/cjs/loader:938:12)
        at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12)
        at node:internal/main/run_main_module:23:47
    
    Node.js v20.5.1
        at makeError (C:\npm\cache\_npx\b66bfc48521ee935\node_modules\execa\lib\error.js:60:11)
        at handlePromise (C:\npm\cache\_npx\b66bfc48521ee935\node_modules\execa\index.js:118:26)
        at processTicksAndRejections (node:internal/process/task_queues:95:5)
        at CorepackExecutor.exec (C:\npm\cache\_npx\b66bfc48521ee935\node_modules\midnight-smoker\src\pm\corepack.ts:50:12)
        at Npm7.pack (C:\npm\cache\_npx\b66bfc48521ee935\node_modules\midnight-smoker\src\pm\npm7.ts:156:20)
        at Smoker.pack (C:\npm\cache\_npx\b66bfc48521ee935\node_modules\midnight-smoker\src\smoker.ts:296:20)
        at Smoker.smoke (C:\npm\cache\_npx\b66bfc48521ee935\node_modules\midnight-smoker\src\smoker.ts:636:34)
        at Object.handler (C:\npm\cache\_npx\b66bfc48521ee935\node_modules\midnight-smoker\src\cli.ts:424:15) {
      shortMessage: 'Command failed with exit code 1: C:\\hostedtoolcache\\windows\\node\\20.5.1\\x64\\node.exe C:\\npm\\cache\\_npx\\b66bfc48521ee935\\node_modules\\.bin\\corepack [email protected] pack --json --pack-destination=C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\midnight-smoker-W4Q28f --foreground-scripts=false',
      command: 'C:\\hostedtoolcache\\windows\\node\\20.5.1\\x64\\node.exe C:\\npm\\cache\\_npx\\b66bfc48521ee935\\node_modules\\.bin\\corepack [email protected] pack --json --pack-destination=C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\midnight-smoker-W4Q28f --foreground-scripts=false',
      escapedCommand: '"C:\\hostedtoolcache\\windows\\node\\20.5.1\\x64\\node.exe" "C:\\npm\\cache\\_npx\\b66bfc48521ee935\\node_modules\\.bin\\corepack" "[email protected]" pack --json "--pack-destination=C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\midnight-smoker-W4Q28f" "--foreground-scripts=false"',
      exitCode: 1,
      signal: undefined,
      signalDescription: undefined,
      stdout: '',
      stderr: 'C:\\npm\\cache\\_npx\\b66bfc48521ee935\\node_modules\\.bin\\corepack:2\r\n' +
        `basedir=$(dirname "$(echo "$0" | sed -e 's,\\\\,/,g')")\r\n` +
        '          ^^^^^^^\r\n' +
        '\r\n' +
        'SyntaxError: missing ) after argument list\r\n' +
        '    at internalCompileFunction (node:internal/vm:73:18)\r\n' +
        '    at wrapSafe (node:internal/modules/cjs/loader:1153:20)\r\n' +
        '    at Module._compile (node:internal/modules/cjs/loader:1197:27)\r\n' +
        '    at Module._extensions..js (node:internal/modules/cjs/loader:1287:10)\r\n' +
        '    at Module.load (node:internal/modules/cjs/loader:1091:32)\r\n' +
        '    at Module._load (node:internal/modules/cjs/loader:938:12)\r\n' +
        '    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12)\r\n' +
        '    at node:internal/main/run_main_module:23:47\r\n' +
        '\r\n' +
        'Node.js v20.5.1',
      failed: true,
      timedOut: false,
      isCanceled: false,
      killed: false
    }
  }
}
Error: Process completed with exit code 1.

Right now, midnight-smoker assumes terminal emoji support. Find a graceful fallback and apply it.

Due to the deprecation of --global-style in npm v9, #242 introduced a version check to avoid triggering the warning. However, the version check just looks at the first character of the output from npm --version and uses --global-style if it is 7 or 8.

Given npm v6 and older is unsupported, I'm not sure we need to worry about it. But it could result in a misleading error message if that's indeed the case. Nevermind that this will break again when npm releases version 70. 😝

I wanted to avoid pulling in all of semver to check the version. Maybe there's something smaller and more robust we could use.

The default package manager is npm@latest, but that should probably be npm@system. That is whatever npm is in your PATH.

Since system is not a dist-tag used by any package managers (I don't love relying on that, but whatever), we'll need to special-case it. In the case of yarn@system, this is whatever yarn you have installed.

If a "system" package manager is not found, throw.

The options to the Smoker class constructor aren't validated with zod nor are they defined as a schema. These objects are slightly different, but we can at least use one to build the other.

Normalization should also occur somewhere via the zod schema(s), as well as the merging of CLI options and config files.

Currently, the rule checks the package root and any directories beneath it except node_modules. It may be useful to allow glob patterns here so the user can tag directories either for inspection or exclusion.

It'd be cool to just work out-of-the-box without a script. Maybe this could act a bit like ESLint where rules could be enabled/disabled/warn

• check if package is importable
• check if it's requireable (if supported)
• #323
• ensure bin is present if declared
• check perms on bin? wontfix (YAGNI)
• #324
• #325
• maybe even check for secrets

some of these may be statically analyzed.

This would check a .d.ts file (the file pointed to by the types field in package.json or the types conditional export(s)) to make sure passes strict mode checks.

I don't think we need to use tsconfig.json to validate this, but we could allow overrides of compilerOptions anyhow.

I tried to disable this in the lint-staged settings but my glob is wrong. __snapshots__/*.js should never be formatted either way.

If we get a 127, it means an executable could not be found. A common cause of this is failing to provide the --add option. Detect this (ideally in a way that's package-manager-agnostic) and display a useful error message to the user.

I want a test that checks some config files against the JSON schema directly.

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: Cannot find preset's package (:sematicCommits). Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

For example: https://github.com/DavidAnson/markdownlint-cli2/actions/runs/5993175027

The same configuration passes on macOS and Ubuntu. --verbose was not enough to expand the nested cause object.

Run npm exec --yes -- midnight-smoker --verbose test-invoke-as-cli
💨 midnight-smoker v6.1.1
- Packing current project…
✖ Package manager "npm" failed to pack
FatalError: midnight-smoker failed unexpectedly
    at Smoker.smoke (C:\npm\cache\_npx\b66bfc48521ee935\node_modules\midnight-smoker\src\smoker.ts:655:13)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Object.handler (C:\npm\cache\_npx\b66bfc48521ee935\node_modules\midnight-smoker\src\cli.ts:423:13) {
  code: 'ESMOKER_FATAL',
  [cause]: {
    error: PackError: Package manager "npm" failed to pack
        at Npm7.pack (C:\npm\cache\_npx\b66bfc48521ee935\node_modules\midnight-smoker\src\pm\npm7.ts:137:13)
        at processTicksAndRejections (node:internal/process/task_queues:95:5)
        at Smoker.pack (C:\npm\cache\_npx\b66bfc48521ee935\node_modules\midnight-smoker\src\smoker.ts:314:20)
        at Smoker.smoke (C:\npm\cache\_npx\b66bfc48521ee935\node_modules\midnight-smoker\src\smoker.ts:618:34)
        at Object.handler (C:\npm\cache\_npx\b66bfc48521ee935\node_modules\midnight-smoker\src\cli.ts:423:13) {
      code: 'ESMOKER_PACK',
      [cause]: [Object]
    }
  }
}
Error: Process completed with exit code 1.

Plugins should be able to export an optional name property. If omitted, the current behavior (look for closest package name) will be used. Plugin names must be unique.

Assert that if there's an engines field in a package.json (probably only the node one for now), and something like exports is provided (without a fallback to main), and the lowest supported version of Node.js understands exports as written.

This would imply that the lowest supported version of Node.js is not actually getting tested in CI.

This rule would check that--unless specified in the os field of package.json--the package is not portable as installed.

This would include things like:

• bash-isms, path expansion or shell scripts in postinstall
• OS-specific packages in dependencies or npm-shrinkwrap.json

This was prompted by a package which published an npm-shrinkwrap.json containing a OS-specific package.

because the load method instantiates npm7 instead.

eliminate load and just use constructor maybe?

This rule would determine the package type (ESM or CJS) via package.json, then check all the sources within to assert they are using the proper extensions.

e.g:

const isESM = isESMPkg(pkgJson);
let cjsExtensions: Set<string>;
let esmExtensions: Set<string>;
if (isESM) {
  cjsExtensions = new Set(['.cjs'])
  esmExtensions = new Set(['.js', '.mjs'])
} else {
  cjsExtensions = new Set(['.js', '.cjs'])
  esmExtensions = new Set(['.mjs'])
}

This would be useful combined with the package manager support in #273.

Instead of pack/install/run, we'd just run the scripts against the package managers. This would further enable the "package manager matrix" idea.

This would check for common/useful fields that aren't in package.json but probably should be. The default severity here should be warn.

• description
• keywords
• author
• repo
• bugs
• homepage
• license

With the addition of support for multiple package managers, we lost the ability to pass extra arguments into the "pack", "install", and "run scripts" steps.

While this may be too awkward to express on the CLI, I could see the pm option allowing not just an array of strings, but an object with the package manager ID as the key, and an {installArgs?: string[], packArgs?: string[], runScriptArgs?: string[]} as the value.

Low priority until someone actually needs it. Some portion of this may be worked around by using package-manager-specific config files.

Add a new option string[] ignore which can be a relative dir path or glob which will cause the rule to not descend into any matching directory.

Motivation: npm gives no warning for stuff in files that doesn't exist

New check that should look at the files field and ensure that if the entry allows a file or glob pattern, the resulting packed package actually matches it. This may be a "deny" pattern, so we can safely ignore those, as we can assume that the package manager won't include such files.

Complicating matters is the existence of an .npmignore (is there a yarn/pnpm equivalent?). Unsure yet what to do here, but the behavior should be researched. Does .npmignore override files or vice-versa? Are they simply merged together?

We may also want to check .npmignore because while it's a deny-first list, it may have negated patterns, just like files.

See #403 for in-progress work. #323 will eventually be part of this.

I want to be able to use AbortSignal and execa only supports that in ESM-only versions. Node supports it natively, so wrapping spawn seems reasonable

To do this, the default Executor would need to use it, as well as any test fixtures or harnesses. Or rather, the default Executor should be the only thing spawning child processes atm. (plugin branch)