Git Product home page Git Product logo

oao's Introduction

oao ๐Ÿ“ฆ Build Status Coverage Status npm version

oao all --parallel

A Yarn-based, opinionated monorepo management tool.

Why? โœจ

  • Works with yarn, hence (relatively) fast!.
  • Simple to use and extend (hope so!).
  • Provides a number of monorepo workflow enhancers: installing all dependencies, adding/removing/upgrading sub-package dependencies, validating version numbers, determining updated sub-packages, publishing everything at once, updating the changelog, etc.
  • Supports yarn workspaces, optimising the monorepo dependency tree as a whole and simplifying bootstrap as well as dependency add/upgrade/remove.
  • Prevents some typical publish errors (using a non-master branch, uncommitted/non-pulled changes).
  • Runs a command or package.json script on all sub-packages, serially or in parallel, optionally following the inverse dependency tree.
  • Provides an easy-to-read, detailed status overview.
  • Support for non-monorepo publishing: benefit from oao's pre-publish checks, tagging, version selection, changelog updates, etc. also in your single-package, non-monorepos.

Assumptions ๐Ÿ’ญ

As stated in the tagline, oao is somewhat opinionated and makes the following assumptions on your monorepo:

  • It uses a synchronized versioning scheme. In other words: a master version is configured in the root-level package.json, and sub-packages will be in sync with that version (whenever they are updated). Some sub-packages can be left behind version-wise if they're not updated, but they'll jump to the master version when they get some love.
  • You use git for version control and have already initialised your repo.
  • Git tags are used for releases (and only for releases), and follow semver: v0.1.3, v2.3.5, v3.1.0-rc.1 and so on.
  • Some sub-packages may be public, others private (flagged "private": true in package.json). OK, no assumption here: rest assured that no private sub-packages will be published by mistake.

Installation

If yarn is not installed in your system, please install it first. If you want to use yarn workspaces (available since yarn 0.28), enable them by running yarn config set workspaces-experimental true and configure the following in your monorepo package.json (replace the glob patterns for your subpackages/workspaces as needed):

"workspaces": [
  "packages/*"
]

Add oao to your development dependencies (use the -W flag to avoid Yarn's warning when installing dependencies on the monorepo root):

$ yarn add oao --dev -W

Usage

To see all CLI options, run oao --help:

Usage: oao [options] [command]

Options:

  -V, --version  output the version number
  -h, --help     output usage information

Commands:

  status [options]                               Show an overview of the monorepo status
  bootstrap [options]                            Install external dependencies and create internal links
  clean [options]                                Delete all node_modules directories from sub-packages and the root package
  add [options] <sub-package> <packages...>      Add dependencies to a sub-package
  remove [options] <sub-package> <packages...>   Remove dependencies from a sub-package
  upgrade [options] <sub-package> [packages...]  Upgrade some/all dependencies of a package
  outdated [options]                             Check for outdated dependencies
  prepublish [options]                           Prepare for a release: validate versions, copy READMEs and package.json attrs
  publish [options]                              Publish updated sub-packages
  reset-all-versions [options] <version>         Reset all versions (incl. monorepo package) to the specified one
  all [options] <command>                        Run a given command on all sub-packages
  run-script [options] <command>                 Run a given script on all sub-packages

You can also get help from particular commands, which may have additional options, e.g. oao publish --help:

Usage: publish [options]

Publish all (non-private) sub-packages

Options:

-s --src <glob>                                           glob pattern for sub-package paths [packages/*] (default: "packages/*")
-i --ignore-src <glob>                                    glob pattern for sub-package paths that should be ignored
-l --link <regex>                                         regex pattern for dependencies that should be linked, not installed
--single                                                  no subpackages, just the root one
--relative-time                                           shorten log dates
--no-master                                               allow publishing from a non-master or non-main branch
--no-check-uncommitted                                    skip uncommitted check
--no-check-unpulled                                       skip unpulled check
--no-checks                                               skip all pre-publish checks
--no-bump                                                 do not increment version numbers (also disables git commit)
--no-confirm                                              do not ask for confirmation before publishing
--no-git-commit                                           skip the commit-tag-push step before publishing
--no-npm-publish                                          skip the npm publish step
--new-version <version>                                   use this version for publishing, instead of asking
--increment-version-by <major|minor|patch|rc|beta|alpha>  increment version by this, instead of asking
--publish-tag <tag>                                       publish with a custom tag (instead of `latest`)
--changelog-path <path>                                   changelog path [CHANGELOG.md] (default: "CHANGELOG.md")
--no-changelog                                            skip changelog updates
--otp <code>                                              use 2-factor authentication to publish your package
--access <type>                                           publish public or restricted packages
-h, --help                                                output usage information

Main commands

In recent versions of npm, remember that you can run oao commands conveniently with the npx tool:

$ npx oao bootstrap
$ npx oao add my-subpackage my-new-dependency --dev
$ npx oao publish

This uses the local oao package inside your monorepo.

oao status

Provides lots of information on the git repo (current branch, last tag, uncommitted/unpulled changes) and subpackage status (version, private flag, changes since last tag, dependencies).

oao status

oao bootstrap

Installs all sub-package dependencies using yarn. External dependencies are installed normally, whereas those belonging to the monorepo itself (and custom links specified with the --link option) are yarn linked. Note that dependencies may end up in different places depending on whether you use yarn workspaces or not (see above).

Development-only dependencies can be skipped by enabling the --production option, or setting the NODE_ENV environment variable to production. Other flags that are passed through to yarn install include --frozen-lockfile, --pure-lockfile and --no-lockfile.

oao clean

Removes node_modules directories from all sub-packages, as well as from the root package.

oao add <sub-package> <deps...>

Adds one or several dependencies to a sub-package. For external dependencies, it passes through yarn add's flags. Internal dependencies are linked. Examples:

$ oao add subpackage-1 jest --dev
$ oao add subpackage-2 react subpackage-1 --exact

oao remove <sub-package> <deps...>

Removes one or several dependencies from a sub-package. Examples:

$ oao remove subpackage-1 jest
$ oao remove subpackage-2 react subpackage-1

oao remove-all <deps...>

Remove one or deveral dependencies from the monorepo (root and subpackages). It automatically runs oao bootstrap after upgrading the package.json files as needed. Examples:

$ oao remove-all leftpad
$ oao remove-all leftpad rightpad centerpad

oao upgrade <sub-package> [deps...]

Upgrade one/several/all dependencies of a sub-package. For external dependencies, it will download the upgraded dependency using yarn. For internal dependencies, it will just update the sub-package's package.json file. Examples:

$ oao upgrade subpackage-1 [email protected]
$ oao upgrade subpackage-2 react [email protected]
$ oao upgrade subpackage-3

oao bump <deps...>

Upgrade one or several dependencies to either their latest version or to a specific version range. In case of internal dependencies, if no version range is given the current version will be used. It automatically runs oao bootstrap after upgrading the package.json files as needed. Examples:

$ oao bump moment
$ oao bump react@^16 react-dom@^16
$ oao bump subpackage-2

oao outdated

Runs yarn outdated on all sub-packages, as well as the root package.

oao prepublish

Carries out a number of chores that are needed before publishing:

  • Checks that all version numbers are valid and <= the master version.
  • Copies <root>/README.md to the main sub-package (the one having the same name as the monorepo).
  • Copies <root>/README-LINK.md to all other sub-packages.
  • Copies several fields from the root package.json to all other package.json files: description, keywords, author, license, homepage, bugs, repository.

oao publish

Carries out a number of steps:

  • Asks the user for confirmation that it has built all sub-packages for publishing (using something like yarn build).
  • Performs a number of checks:
    • The current branch should be master or main.
    • No uncommitted changes should remain in the working directory.
    • No unpulled changes should remain.
  • Determines which sub-packages need publishing (those which have changed with respect to the last tagged version).
  • Asks the user for an incremented master version (major, minor, patch or pre-release major), that will be used for the root package as well as all updated sub-packages.
  • Asks the user for final confirmation before publishing.
  • Updates versions in package.json files, commits the updates, adds a tag and pushes all the changes.
  • Publishes updated sub-packages.

There are lots of custom options for oao publish. Chances are, you can disable each one of the previous steps by means of one of those options. Check them all with oao publish --help.

Note: There is a problem when running oao publish as a script run with yarn. As a workaround, either run oao publish manually from the command line, or put it in a script and run it with npm, not yarn.

oao all <command>

Executes the specified command on all sub-packages (private ones included), with the sub-package's root as current working directory. Examples:

$ oao all ls
$ oao all "ls -al"
$ oao all "yarn run compile"
$ oao all --tree "yarn run compile"

By default, oao all runs sequentially. Sometimes you must run commands in parallel, for example when you want to compile all sub-packages with a watch option:

$ oao all "yarn run compileWatch" --parallel

Note: some terminals may have problems with parallel logs (based on terminal-kit). If you experience issues, use the --no-parallel-logs flag. If you're using the default terminal or Hyper on OS X or Windows, you should be fine.

Use --tree if you want to follow the inverse dependency tree (starting from the tree leaves).

You can also pass extra arguments to the command separating them with a --: oao all ls -- -al is equivalent to oao all 'ls -al'. This can be useful for adding extra commands to scripts in package.json.

oao run-script <script>

Similar to oao all <command>, it executes the specified (package) script on all sub-packages. Missing scripts will be skipped. Examples:

$ oao run-script start
$ oao run-script start --parallel
$ oao run-script start --tree

By default, oao run-script runs sequentially. Use --parallel to run the scripts in parallel, and --tree if you want to follow the inverse dependency tree (starting from the tree leaves).

You can also run all scripts matching a given glob pattern: oao run-script test:*.

Credits ๐Ÿ‘

  • lerna: for general inspiration.
  • yarn: for a fast, secure and reliable way to do dependency management.
  • np: for the prepublish checks.

Why oao? ๐Ÿ˜Ž

Basically, many other names I could come up with were either too boring (mono-repo), or already taken. oao stands for one and only ๐Ÿ˜ฌ, which is a reference to the individual nature of monorepos, as well as a beautiful song by Adele. Yes, I agree it's far-fetched, but extremely short and convenient!

Changelog ๐Ÿ“œ

License (MIT) ๐Ÿ“š

Copyright (c) Guillermo Grau Panea 2017-now

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

oao's People

Contributors

aghassi avatar eliperelman avatar gordlea avatar guigrpa avatar jeroenptrs avatar jogelin avatar jozanza avatar mdreizin avatar otariidae avatar pedronauck avatar simenb avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

oao's Issues

status command throws "Cannot read property 'length' of undefined"

Currently status command throws the following error:

TypeError: Cannot read property 'length' of undefined
    at field (/Users/mdreizin/.config/yarn/global/node_modules/oao/lib/status.js:227:10)
    at _callee3$ (/Users/mdreizin/.config/yarn/global/node_modules/oao/lib/status.js:177:51)
    at tryCatch (/Users/mdreizin/.config/yarn/global/node_modules/regenerator-runtime/runtime.js:64:40)
    at Generator.invoke [as _invoke] (/Users/mdreizin/.config/yarn/global/node_modules/regenerator-runtime/runtime.js:299:22)
    at Generator.prototype.(anonymous function) [as next] (/Users/mdreizin/.config/yarn/global/node_modules/regenerator-runtime/runtime.js:116:21)
    at step (/Users/mdreizin/.config/yarn/global/node_modules/oao/lib/status.js:13:191)
    at /Users/mdreizin/.config/yarn/global/node_modules/oao/lib/status.js:13:361

Need to support scoped modules

When running oao bootstrap an error is thrown:

Error: Package name does not match directory name

This is due to a package module having a scoped name versus the folder name

// package name
@scopedname/module-name

// folder name
scopedname-module-name

reset-all-versions could also change the cross dependencies version

Hello,

In case of having cross dependencies between packages, what do you think about updating also the version of those dependencies ?

BEFORE
packages/lib1/package.json:

{
    "name": "lib-1",
    "version":"0.1.0"
}

packages/lib2/package.json:

{
    "name": "lib-2",
    "version":"0.1.0",
    "peerDependencies": {
        "lib-1": "0.1.0"
    }
}

RESET ALL VERSION

$ oao reset-all-versions 0.2.0

AFTER
packages/lib1/package.json:

{
    "name": "lib-1",
    "version":"0.2.0"
}

packages/lib2/package.json:

{
    "name": "lib-2",
    "version":"0.2.0",
    "peerDependencies": {
        "lib-1": "0.1.0" ---> THIS IS NOT UPDATED (same for dependencies or devDependencies)
    }
}

What do you think ?

oao yarn invocation errors often give insufficient information

In #51, the yarn link neutrino command failed with:

2017-10-10T14:09:45.076Z                 main TRACE โ”Œโ”€โ”€ Run cmd yarn link neutrino at packages/neutrino-middleware-env [CREATED]
2017-10-10T14:10:03.867Z           storyboard WARN  โ””โ”€โ”€ REVEALED PAST LOGS END HERE
2017-10-10T14:10:03.867Z                 yarn ERROR | Killed
2017-10-10T14:10:03.868Z                 yarn INFO  | 
2017-10-10T14:10:04.008Z                 main ERROR Command 'yarn link neutrino' failed at packages/neutrino-middleware-env
                                         main ERROR   name: 'Error'
                                         main ERROR   message: Command failed: /bin/sh -c yarn link neutrino
                                         main ERROR   message: Killed
                                         main ERROR   message: 
                                         main ERROR   stack: Error: Command failed: /bin/sh -c yarn link neutrino
                                         main ERROR   stack: Killed
                                         main ERROR   stack: 
                                         main ERROR   stack:     at Promise.all.then.arr (/home/travis/build/edmorley/neutrino-dev/node_modules/oao/node_modules/execa/index.js:210:11)
                                         main ERROR   stack:     at process._tickCallback (internal/process/next_tick.js:109:7)
2017-10-10T14:10:07.704Z                 main TRACE โ””โ”€โ”€ Run cmd yarn link neutrino at packages/neutrino-middleware-env [CLOSED]
Error: Command failed: yarn link neutrino
    at _callee2$ (/home/travis/build/edmorley/neutrino-dev/node_modules/oao/lib/utils/shell.js:173:19)
    at tryCatch (/home/travis/build/edmorley/neutrino-dev/node_modules/regenerator-runtime/runtime.js:65:40)
    at GeneratorFunctionPrototype.invoke [as _invoke] (/home/travis/build/edmorley/neutrino-dev/node_modules/regenerator-runtime/runtime.js:303:22)
    at GeneratorFunctionPrototype.prototype.(anonymous function) [as throw] (/home/travis/build/edmorley/neutrino-dev/node_modules/regenerator-runtime/runtime.js:117:21)
    at step (/home/travis/build/edmorley/neutrino-dev/node_modules/oao/lib/utils/shell.js:28:191)
    at /home/travis/build/edmorley/neutrino-dev/node_modules/oao/lib/utils/shell.js:28:402
    at process._tickCallback (internal/process/next_tick.js:109:7)
2017-10-10T14:10:07.712Z           storyboard INFO  โ””โ”€โ”€ ROOT STORY: Node.js 6.11.4 on Linux 64-bit, SB 3.1.3 [CLOSED]

This error message doesn't really help with debugging the problem, since:

  • What does "Killed" mean? Did the yarn command hang and the oxo exec call have to kill it? If so ideally there would be output explaining this.
  • Where is the stdout/stderr from yarn? Even if the command timed out, there should still be the header row from yarn, unless output buffering is causing issues?
  • If yarn didn't time out, but just gave an unhelpful stdout/stderr, then perhaps the exit code should be shown? (Maybe only if >1 ?)
  • In what directory was the command being run? The log says "Command ... failed at packages/neutrino-middleware-env", however I presume that's not the real directory, given how yarn link is supposed to work? Knowing how to run the command exactly as oao is, will help with reproducing.
  • There's quite a bit of duplication between the "main ERROR" prefixed lines and the non-storyboard prefixed lines later.

Many thanks :-)

Mercurial Support

Hello,

would you be interested by a pull-request for mercurial support ?

INVALID_DIR_NAME

Ran oao for the first time.

oao status
Error: INVALID_DIR_NAME
    at validatePkgName (/Users/burdon/.config/yarn/global/node_modules/oao/lib/utils/readSpecs.js:88:11)
    at readOneSpec (/Users/burdon/.config/yarn/global/node_modules/oao/lib/utils/readSpecs.js:75:3)
    at /Users/burdon/.config/yarn/global/node_modules/oao/lib/utils/readSpecs.js:46:25
    at Array.forEach (native)
    at _callee$ (/Users/burdon/.config/yarn/global/node_modules/oao/lib/utils/readSpecs.js:45:22)
    at tryCatch (/Users/burdon/.config/yarn/global/node_modules/regenerator-runtime/runtime.js:65:40)
    at Generator.invoke [as _invoke] (/Users/burdon/.config/yarn/global/node_modules/regenerator-runtime/runtime.js:303:22)
    at Generator.prototype.(anonymous function) [as next] (/Users/burdon/.config/yarn/global/node_modules/regenerator-runtime/runtime.js:117:21)
    at step (/Users/burdon/.config/yarn/global/node_modules/oao/lib/utils/readSpecs.js:24:191)
    at /Users/burdon/.config/yarn/global/node_modules/oao/lib/utils/readSpecs.js:24:361

oao equivalent of yarn upgrade-interactive?

Both yarn outdated and yarn upgrade-interactive return Outdated lockfile. Please run yarn install and try again., likely do to the oao dependencies.

oao upgrade seems to bail with an error on the 1st outdated package.

 โฏ oao all "yarn upgrade-interactive"
2017-08-30T15:44:00.857Z           storyboard INFO  โ”Œโ”€โ”€ ROOT STORY: null, SB 3.1.1 [CREATED]
2017-08-30T15:44:00.858Z           storyboard INFO  Log filter: *:DEBUG
2017-08-30T15:44:01.168Z                 main INFO  โ”Œโ”€โ”€ Run cmd yarn upgrade-interactive at examples/with-404 [CREATED]
2017-08-30T15:44:01.393Z                 yarn INFO  yarn upgrade-interactive v0.27.5
2017-08-30T15:44:01.527Z                 yarn ERROR error Outdated lockfile. Please run `yarn install` and try again.
2017-08-30T15:44:01.527Z                 yarn INFO  info Visit https://yarnpkg.com/en/docs/cli/upgrade-interactive for documentation about this command.
2017-08-30T15:44:01.537Z                 yarn INFO
2017-08-30T15:44:01.541Z                 main ERROR Command 'yarn upgrade-interactive' failed at examples/with-404
                                         main ERROR   name: 'Error'
                                         main ERROR   message: Command failed: /bin/sh -c yarn upgrade-interactive
                                         main ERROR   message: error Outdated lockfile. Please run `yarn install` and try again.
                                         main ERROR   message: yarn upgrade-interactive v0.27.5
                                         main ERROR   message: info Visit https://yarnpkg.com/en/docs/cli/upgrade-interactive for documentation about this command.
                                         main ERROR   message:
                                         main ERROR   stack: Error: Command failed: /bin/sh -c yarn upgrade-interactive
                                         main ERROR   stack: error Outdated lockfile. Please run `yarn install` and try again.
                                         main ERROR   stack: yarn upgrade-interactive v0.27.5
                                         main ERROR   stack: info Visit https://yarnpkg.com/en/docs/cli/upgrade-interactive for documentation about this command.
                                         main ERROR   stack:
                                         main ERROR   stack:     at Promise.all.then.arr (/Users/eric/.config/yarn/global/node_modules/execa/index.js:210:11)
                                         main ERROR   stack:     at process._tickCallback (internal/process/next_tick.js:109:7)
2017-08-30T15:44:01.543Z                 main INFO  โ””โ”€โ”€ Run cmd yarn upgrade-interactive at examples/with-404 [CLOSED]

Unable to install new sub-package dependency

When trying to install a new dependency in a sub-package, yarn complains that it is unable to find a version matching a linked dependency. For example:

Monorepo is at v1.0.0. We start development on major version 2.0.0. Package sub-b depends on sub-a. When you try to yarn add anything new in sub-b, yarn will attempt to fix node_modules up by putting the correct version of sub-a 2.0.0 in, but it can't because it doesn't exist yet. Calamity ensues.

As a workaround I used npm to save the new dependency, deleted the yarn.lock files, and re-ran oao bootstrap.

run-script in dependency order

As far as I can see, the thing keeping us from switching from lerna to oao is that oao doesn't have the ability to run scripts in dependency order, which is something lerna does by default. Basically, this means that it will run scripts in packages with no internal dependencies first, and then outwards from there. This is something generally needed to compile typescript inside of a mono-repo.

Add a `run` command to the CLI

Hey @guigrpa - Awesome job with this CLI - Really enjoying it so far! :)

The only thing I currently am missing is a run command similar to the lerna run command. Basically it would iterate through all packages, check if the provided script name is within the package.json and if that is the case it would execute a yarn run <script-name>.

Would you be up for such a contribution? I would be willing to do it in the next few days and create a PR to this repository! :)

oao docker issue when bootstrap with TypeError: cmd.opts is not a function

Hello,

I would like to use oao during my build inside a docker image but I have an issue during the bootstrap :

16:36:59.680 $ oao bootstrap -s ./packages/*/ 
16:36:59.985 2017-02-22T15:36:59.974Z           storyboard INFO  ??? ROOT STORY: Node.js 7.5.0 on Linux 64-bit, SB 3.0.0 [CREATED]
16:36:59.986 2017-02-22T15:36:59.975Z           storyboard INFO  Log filter: *:DEBUG
16:37:00.174 2017-02-22T15:37:00.173Z           storyboard INFO  ??? ROOT STORY: Node.js 7.5.0 on Linux 64-bit, SB 3.0.0 [CLOSED]
16:37:00.174 /home/app/node_modules/oao/lib/index.js:60
16:37:00.174   return (0, _bootstrap2.default)(cmd.opts());
16:37:00.174                                       ^
16:37:00.174 
16:37:00.174 TypeError: cmd.opts is not a function
16:37:00.174     at Command.<anonymous> (/home/app/node_modules/oao/lib/index.js:60:39)
16:37:00.174     at Command.listener (/home/app/node_modules/commander/index.js:301:8)
16:37:00.174     at emitTwo (events.js:106:13)
16:37:00.175     at Command.emit (events.js:192:7)
16:37:00.175     at Command.parseArgs (/home/app/node_modules/commander/index.js:615:12)
16:37:00.175     at Command.parse (/home/app/node_modules/commander/index.js:458:21)
16:37:00.175     at Object.<anonymous> (/home/app/node_modules/oao/lib/index.js:98:21)
16:37:00.175     at Module._compile (module.js:571:32)
16:37:00.175     at Object.Module._extensions..js (module.js:580:10)
16:37:00.175     at Module.load (module.js:488:32)
16:37:00.181 error Command failed with exit code 1.
16:37:00.181 info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
16:37:00.186 ERROR: "bootstrap" exited with 1.
16:37:00.192 error Command failed with exit code 1.

My docker-compose.yml :

version: '2'
services:
  js-build:
    container_name: js-build
    image: node:7.5.0
    command: npm install yarn -g --progress=false && npm install && yarn build"
    working_dir: /home/app
    volumes:
      - .:/home/app

You can run it using the command :

docker-compose run js-build

Link sub-package in root?

My current setup:

/packages
  /project-core
    /bin
      /start
package.json

Is there a way for the package-core project (which references "bin": "./bin/start") to be "linked" or installed in a way as part of the root project?

The ideal being:

{
  "dependencies": {
    "project-core": "*"
  },
  "scripts": {
    "start": "project-core"
  }
}

REVEALED PAST LOGS END HERE appears before the error is displayed

Hello,

For the package lib-2, the "revealed end logs" line appears before the error is displayed :

> oao bootstrap -s ./packages/*/

2017-02-21T08:06:48.363Z           storyboard INFO  โ”Œโ”€โ”€ ROOT STORY: Node.js 7.5.0 on Win32 64-bit, SB 3.0.0 [CREATED]
2017-02-21T08:06:48.364Z           storyboard INFO  Log filter: *:DEBUG
2017-02-21T08:06:48.549Z                 main INFO  Reading all package.json files...
2017-02-21T08:06:48.551Z                 main INFO  PASS 1: processing @bamaa/lib-1...
2017-02-21T08:06:48.552Z                 main INFO    - Registering...
2017-02-21T08:06:48.924Z                 main INFO    - Installing external dependencies...
2017-02-21T08:06:49.320Z                 main INFO  PASS 1: processing @bamaa/lib-2...
2017-02-21T08:06:49.321Z                 main INFO    - Registering...
2017-02-21T08:06:49.681Z                 main INFO    - Installing external dependencies...
2017-02-21T08:06:50.176Z           storyboard WARN  โ”Œโ”€โ”€ REVEALED PAST LOGS BEGIN HERE (due to warning/error)
2017-02-21T08:06:49.682Z                 main TRACE โ”Œโ”€โ”€ Run cmd yarn install at ./packages/lib-2/ [CREATED]
2017-02-21T08:06:49.970Z                 yarn INFO  | yarn install v0.20.3
2017-02-21T08:06:50.040Z                 yarn INFO  | [1/4] Resolving packages...
2017-02-21T08:06:50.141Z                 yarn INFO  | [2/4] Fetching packages...
2017-02-21T08:06:50.175Z                 yarn INFO  | [3/4] Linking dependencies...
2017-02-21T08:06:50.178Z           storyboard WARN  โ””โ”€โ”€ REVEALED PAST LOGS END HERE
2017-02-21T08:06:50.176Z                 yarn ERROR | warning "[email protected]" has unmet peer dependency "@angular/core@^2.0.0".
2017-02-21T08:06:50.178Z                 yarn ERROR | warning "[email protected]" has unmet peer dependency "@angular/http@^2.0.0".
2017-02-21T08:06:50.288Z                 yarn INFO  | [4/4] Building fresh packages...
2017-02-21T08:06:50.297Z                 yarn INFO  | Done in 0.34s.
2017-02-21T08:06:50.315Z                 yarn INFO  |
2017-02-21T08:06:50.317Z                 main TRACE โ””โ”€โ”€ Run cmd yarn install at ./packages/lib-2/ [CLOSED]
2017-02-21T08:06:50.318Z                 main INFO  PASS 1: processing @bamaa/lib-3...
2017-02-21T08:06:50.318Z                 main INFO    - Registering...
2017-02-21T08:06:50.680Z                 main INFO    - Installing external dependencies...
2017-02-21T08:06:51.061Z                 main INFO  PASS 2: installing internal deps for @bamaa/lib-1...
2017-02-21T08:06:51.061Z                 main INFO  PASS 2: installing internal deps for @bamaa/lib-2...
2017-02-21T08:06:51.062Z                 main INFO  PASS 2: installing internal deps for @bamaa/lib-3...
2017-02-21T08:06:51.062Z                 main INFO    - Linking to @bamaa/lib-1...
2017-02-21T08:06:51.425Z                 main INFO    - Linking to @bamaa/lib-2...
2017-02-21T08:06:51.803Z           storyboard INFO  โ””โ”€โ”€ ROOT STORY: Node.js 7.5.0 on Win32 64-bit, SB 3.0.0 [CLOSED]

Run script in sub-package from root package

What is the best way to run a script in a sub-package from the root-package?

Sub-package /packages/app/package.json:

  "scripts": {
      "start": "neutrino start"
  }

Root-package /package.json:

  "scripts": {
      "start": "oao app start"
  }

Obviously, that doesn't work. But is there something like that I'm not aware of?

Getting started?

Do I steal lerna.json from another project, or are there some commands I can run?

reset version of a library on all packages

Hello,

Sometimes, in a monorepo, it could be good to change the version of a library in all packages. For example, if packages uses Angular or React, we would like to be sure that all packages are sharing the same version.

I think there are different ways of doing :

Having a command that set a version for a dependency, for example:

$ oao set-version @angular 4.0.0 --> all dependencies of this scope are updated
$ oao set-version @angular/core,@angular/http 4.0.0 --> only core and http are updated

Copy versions from root/parent :

In my case it is what I am doing, I don't like my solution :

BEFORE
package.json:

{
    "name": "root",
    "version":"0.1.0"
    "peerDependenciesPlaceholders": {
        "0.0.0-ANGULAR_VERSION_PLACEHOLDER": "^4.0.0-rc.1"
    }
}

packages/lib1/package.json:

{
    "name": "lib-1",
    "version":"0.1.0",
    "peerDependencies": {
        "@angular/core": "0.0.0-ANGULAR_VERSION_PLACEHOLDER",
    }
}

RUN INTERNAL SCRIPT
I run internal script that replaces placeholders on all packages corresponding to the placeholders found in peerDependenciesPlaceholders of the root package.json

AFTER
packages/lib1/package.json:

{
    "name": "lib-1",
    "version":"0.1.0",
    "peerDependencies": {
        "@angular/core": "^4.0.0-rc.1"
    }
}

Other solutions ?

Ideas for building docker images from individual packages

I've been experimenting with oao and liking the simple workflow. One major hiccup I've hit is that it is difficult to effectively build docker images out of one of the monorepo's packages when it depends on another private: true package.

For local testing, the linking solution works quite well. However, when it comes time to build out images, I'm hitting a major workflow barrier. It boils down to somehow getting the private, local packages into the docker build context in a way that yarn can read them.

Any ideas?

oao status fails if upstream is not set

What's weird is that I've used git for years, have a project we've been working on for ~2 years, and this command failed:

$ oao status

It's due to this command here:

const { stdout } = await exec('git rev-list --count --left-only @{u}...HEAD', { logLevel: 'trace' });

I was able to fix it with this:

$ git push --set-upstream origin master

--version during publish

Hello,

If the CI environment is reponsible on the publishing, it should be able to provide the version in command line.

$ oao publish --version 0.1.0-rc.0

Null story board node tag

Hello folks! I started using oao and enjoying it so far.

A minor thing is that I don't see the Node ... thing in the console output:

2017-07-12T10:01:01.759Z           storyboard INFO  โ”Œโ”€โ”€ ROOT STORY: null, SB 3.1.1 [CREATED]

The only thing that I have to add here is that I am using nvm for managing the Node version.

Thanks for your work on oao !

oao bootstrap fails under node 6 and yarn 1.0.0+ with "Killed" error during yarn link

STR:

  1. Download this testcase: https://github.com/edmorley/neutrino-dev/archive/testcase-oao-yarn-link-killed.zip
  2. Push to a new test repo that has Travis enabled
  3. Whilst waiting for the jobs to complete, edit .travis.yml, replacing the yarn install step --version 1.0.0 with --version 0.28.4
  4. Push to the repo again

Expected:
All four Travis jobs pass (each push triggers jobs on 2 node versions).

Actual:

Worth noting:

  • This is using oao 0.10.3 (see testcase's package.json / yarn.lock).
  • Yarn 1.0.0 is the first release to break, 0.28.4 the last working. This still reproduces using latest yarn too (1.2.0).
  • Travis caching has been disabled, so there are no node_modules present, and the yarn cache is empty.
  • In the .travis.yml I'm running oao directly rather than via a yarn run command, to rule out any leftover YARN_SILENT issues, however there is still no useful stdout/stderr from the failed yarn link command (I'll file a new issue for adding more debugging info for cases like these).
  • There were sadly no yarn-error.log files created that might have shed some light on the error.
  • I haven't attempted to repro locally, since switching node versions is a pain on Windows.

The yarn 1.0.0 release notes are here:
https://github.com/yarnpkg/yarn/releases/tag/v1.0.0

Bootstrap takes 60 seconds to run even when nothing has changed

STR:

  1. Install nodejs 8.1.0 and yarn v0.24.6 under Ubuntu 16.04.
  2. git clone https://github.com/mozilla-neutrino/neutrino-dev
  3. cd neutrino-dev && git checkout 13adea32550de2f1eae9ae79f2980f8a2f6b5e82
  4. yarn install (this installs oao v0.8.0)
  5. yarn bootstrap (which runs oao bootstrap from the local node_modules)
  6. yarn bootstrap

Expected:
The second time bootstrap is run, the duration is significantly less than the first, ideally <5-10s.

Actual:
The first bootstrap took 125 seconds (see https://emorley.pastebin.mozilla.org/9024429), the second still took 63 seconds (see https://emorley.pastebin.mozilla.org/9024430), even though it had nothing to do.

This also affects us on Travis, since it means caching the sub-package's node_modules gives little time saving (and in fact quite a disadvantage, given travis-ci/travis-ci#7898) - see neutrinojs/neutrino#248.

Do you have any ideas as to what might be causing this?

Many thanks!

Linking of unsatisfied local versions.

Given this scenario:

- packages/[email protected]
- packages/app-one
    - deps: [email protected]
- packages/app-two
    - deps: [email protected]

The default behavior seems to be to link [email protected] into both app-one and app-two, even though app-one is only satisfied by 1.0.0. I'm a bit surprised this is happening (and that it also seems to be the case in lerna), as I would expect oao to try to fetch a version of module-one that satisfies its package.json instead once it notices that the local version does not. Right now you could end up with hard to track issues during development if one of the apps is using an incompatible version, especially since this also does not seem to be picked up by oao outdated (which seems to be because yarn outdated does not like symlinked modules). Is there any reasononing behind this behavior?

I guess this is also partly covered by #16 and the yarn knit discussion, but would there perhaps be a workaround that can be used in the meantime? Could oao outdated for example warn for version inconsistencies in consuming packages so that people running a publish are at least aware of an incompatibility?

Authentication errors when trying to publish

For some reason I always get an authentication error when trying to use oao publish. When that command fails, if I use oao all "npm publish", everything publishes fine after that.

yarn logs
โฏ yarn release
yarn release v0.20.3
$ yarn release:pre && oao publish
yarn release:pre v0.20.3
$ oao prepublish
2017-02-17T16:10:05.512Z           storyboard INFO  โ”Œโ”€โ”€ ROOT STORY: Node.js 6.9.1 on Darwin 64-bit, SB 3.0.0-rc.2 [CREATED]
2017-02-17T16:10:05.512Z           storyboard INFO  Log filter: *:DEBUG
2017-02-17T16:10:05.646Z                 main INFO  Reading all package.json files...
2017-02-17T16:10:05.649Z                 main DEBUG Copying README-LINK.md -> packages/neutrino/README.md...
cp: no such file or directory: README-LINK.md
2017-02-17T16:10:05.653Z                 main DEBUG Copying README-LINK.md -> packages/neutrino-preset-airbnb-base/README.md...
cp: no such file or directory: README-LINK.md
2017-02-17T16:10:05.653Z                 main DEBUG Copying README-LINK.md -> packages/neutrino-preset-jest/README.md...
cp: no such file or directory: README-LINK.md
2017-02-17T16:10:05.654Z                 main DEBUG Copying README-LINK.md -> packages/neutrino-preset-karma/README.md...
cp: no such file or directory: README-LINK.md
2017-02-17T16:10:05.654Z                 main DEBUG Copying README-LINK.md -> packages/neutrino-preset-mocha/README.md...
cp: no such file or directory: README-LINK.md
2017-02-17T16:10:05.655Z                 main DEBUG Copying README-LINK.md -> packages/neutrino-preset-node/README.md...
cp: no such file or directory: README-LINK.md
2017-02-17T16:10:05.655Z                 main DEBUG Copying README-LINK.md -> packages/neutrino-preset-react/README.md...
cp: no such file or directory: README-LINK.md
2017-02-17T16:10:05.656Z                 main DEBUG Copying README-LINK.md -> packages/neutrino-preset-web/README.md...
cp: no such file or directory: README-LINK.md
2017-02-17T16:10:05.658Z                 main INFO  Updating package attributes
                                         main INFO    description: undefined
                                         main INFO    keywords: undefined
                                         main INFO    author: 'Eli Perelman <[email protected]>'
                                         main INFO    license: 'MPL-2.0'
                                         main INFO    homepage: 'https://neutrino.js.org'
                                         main INFO    bugs: 'https://github.com/mozilla-neutrino/neutrino-dev/issues'
                                         main INFO    repository: 'mozilla-neutrino/neutrino-dev'
2017-02-17T16:10:05.662Z                 main WARN  Please make sure you commit all changes before you attempt "oao publish"
2017-02-17T16:10:05.663Z           storyboard INFO  โ””โ”€โ”€ ROOT STORY: Node.js 6.9.1 on Darwin 64-bit, SB 3.0.0-rc.2 [CLOSED]
โœจ  Done in 0.56s.
2017-02-17T16:10:05.950Z           storyboard INFO  โ”Œโ”€โ”€ ROOT STORY: Node.js 6.9.1 on Darwin 64-bit, SB 3.0.0-rc.2 [CREATED]
2017-02-17T16:10:05.951Z           storyboard INFO  Log filter: *:DEBUG
2017-02-17T16:10:06.102Z                 main INFO  Reading all package.json files...
? Have you built all your packages for production? Yes
2017-02-17T16:10:09.326Z                 main INFO  Current branch: master
2017-02-17T16:10:09.343Z                 main INFO  No uncommitted changes
2017-02-17T16:10:09.356Z                 main INFO  Remote history matches local history
2017-02-17T16:10:09.401Z                 main INFO  - Package neutrino (currently 4.0.0) has changed (#files: 1)
2017-02-17T16:10:09.414Z                 main INFO  - Package neutrino-preset-airbnb-base (currently 4.0.0) has changed (#files: 1)
2017-02-17T16:10:09.428Z                 main INFO  - Package neutrino-preset-jest (currently 4.0.0) has changed (#files: 1)
2017-02-17T16:10:09.444Z                 main INFO  - Package neutrino-preset-karma (currently 4.0.0) has changed (#files: 1)
2017-02-17T16:10:09.457Z                 main INFO  - Package neutrino-preset-mocha (currently 4.0.0) has changed (#files: 1)
2017-02-17T16:10:09.471Z                 main INFO  - Package neutrino-preset-node (currently 4.0.0) has changed (#files: 1)
2017-02-17T16:10:09.486Z                 main INFO  - Package neutrino-preset-react (currently 4.0.0) has changed (#files: 2)
2017-02-17T16:10:09.500Z                 main INFO  - Package neutrino-preset-web (currently 4.0.0) has changed (#files: 1)
2017-02-17T16:10:09.502Z                 main INFO  Last tag found: v4.0.0
? Current version is 4.0.0. Next one? Patch (4.0.1)
? Confirm publish (8 package/s, v4.0.1)? Yes
2017-02-17T16:10:25.580Z                 main INFO  โ”Œโ”€โ”€ Run cmd npm publish at packages/neutrino [CREATED]
2017-02-17T16:10:27.448Z                  npm ERROR | npm ERR! publish Failed PUT 401
2017-02-17T16:10:27.450Z                  npm ERROR | npm ERR! Darwin 16.4.0
2017-02-17T16:10:27.450Z                  npm ERROR | npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "publish"
2017-02-17T16:10:27.450Z                  npm ERROR | npm ERR! node v6.9.1
2017-02-17T16:10:27.450Z                  npm ERROR | npm ERR! npm  v4.1.2
2017-02-17T16:10:27.450Z                  npm ERROR | npm ERR! code E401
2017-02-17T16:10:27.453Z                  npm ERROR | npm ERR! Could not authenticate eliperelman : neutrino
2017-02-17T16:10:27.453Z                  npm ERROR | npm ERR!
2017-02-17T16:10:27.453Z                  npm ERROR | npm ERR! If you need help, you may report this error at:
2017-02-17T16:10:27.453Z                  npm ERROR | npm ERR!     <https://github.com/npm/npm/issues>
2017-02-17T16:10:27.459Z                  npm ERROR | npm ERR! Please include the following file with any support request:
2017-02-17T16:10:27.459Z                  npm ERROR | npm ERR!     /Users/eli/code/neutrino-dev/packages/neutrino/npm-debug.log
2017-02-17T16:10:27.467Z                  npm INFO  |
2017-02-17T16:10:27.468Z                 main ERROR Command failed [1]
2017-02-17T16:10:27.469Z                 main INFO  โ””โ”€โ”€ Run cmd npm publish at packages/neutrino [CLOSED]
(node:74967) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: Command failed: npm publish
2017-02-17T16:10:27.471Z           storyboard INFO  โ””โ”€โ”€ ROOT STORY: Node.js 6.9.1 on Darwin 64-bit, SB 3.0.0-rc.2 [CLOSED]
โœจ  Done in 22.69s.

DeprecationWarning: Unhandled promise rejections are deprecated

When a command failed to run, there is a waning about the promise rejection (node 7.5.0 on windows 10) :

(node:11656) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: Command failed: tsc
(node:11656) DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero
 exit code.

Full log :

$ oao all tsc
2017-02-20T14:31:48.728Z           storyboard INFO  โ”Œโ”€โ”€ ROOT STORY: Node.js 7.5.0 on Win32 64-bit, SB 3.0.0 [CREATED]
2017-02-20T14:31:48.729Z           storyboard INFO  Log filter: *:DEBUG
2017-02-20T14:31:48.921Z                 main INFO  โ”Œโ”€โ”€ Run cmd tsc at packages/lib-1 [CREATED]
2017-02-20T14:31:50.119Z                  tsc INFO  | 
2017-02-20T14:31:50.122Z                 main INFO  โ””โ”€โ”€ Run cmd tsc at packages/lib-1 [CLOSED]
2017-02-20T14:31:50.123Z                 main INFO  โ”Œโ”€โ”€ Run cmd tsc at packages/lib-2 [CREATED]
2017-02-20T14:31:51.551Z                  tsc INFO  | 
2017-02-20T14:31:51.552Z                 main INFO  โ””โ”€โ”€ Run cmd tsc at packages/lib-2 [CLOSED]
2017-02-20T14:31:51.552Z                 main INFO  โ”Œโ”€โ”€ Run cmd tsc at packages/lib-3 [CREATED]
2017-02-20T14:31:52.764Z                  tsc INFO  | src/lib-3.module.ts(12,8): error TS2307: Cannot find module 'ng2-translate'.
2017-02-20T14:31:52.779Z                  tsc INFO  | 
2017-02-20T14:31:52.779Z                 main ERROR Command failed [2]
2017-02-20T14:31:52.780Z                 main INFO  โ””โ”€โ”€ Run cmd tsc at packages/lib-3 [CLOSED]
(node:11656) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: Command failed: tsc
(node:11656) DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero
 exit code.
2017-02-20T14:31:52.782Z           storyboard INFO  โ””โ”€โ”€ ROOT STORY: Node.js 7.5.0 on Win32 64-bit, SB 3.0.0 [CLOSED]
Done in 4.68s.

By the way, maybe you can help, when I want to compile lib-3 which have a dependency graph like :

(internal) lib-3 -> (internal) lib-1 -> (external) ng2-translate

I doesn't fin the ng2-translate but it's a dependency of lib-1 not of lib-3...probably caused by the link...(sorry for this stupid question)

logger produces a lot of empty lines in Terminal/iTerm

Logger produces a lot of empty lines in Terminal/iTerm.

I have tested mot popular terminals and it works correctly in Hyper:

  • Terminal
  • iTerm
  • Hyper
yarn run v0.20.3
$ oao all "yarn run build" --parallel 
2017-02-23T09:01:53.343Z           storyboard INFO  โ”Œโ”€โ”€ ROOT STORY: Node.js 7.5.0 on Darwin 64-bit, SB 3.0.0 [CREATED]
2017-02-23T09:01:53.344Z           storyboard INFO  Log filter: *:DEBUG

<a lot of empty lines>

 1  ROOT STORY: Node.js 7.5.0 on Darwin 64-bit, SB 3.0.0
  2017-02-23T09:01:53.344Z           storyboard INFO  Log filter: *:DEBUG

<a lot of empty lines>

 1  ROOT STORY: Node.js 7.5.0 on Darwin 64-bit, SB 3.0.0
  2017-02-23T09:01:53.344Z           storyboard INFO  Log filter: *:DEBUG

<a lot of empty lines>

 2  Run cmd yarn run build at packages/app

<a lot of empty lines>

 1  ROOT STORY: Node.js 7.5.0 on Darwin 64-bit, SB 3.0.0
  2017-02-23T09:01:53.344Z           storyboard INFO  Log filter: *:DEBUG

<a lot of empty lines>

 2  Run cmd yarn run build at packages/app

<a lot of empty lines>

 3  Run cmd yarn run build at packages/app-config

<a lot of empty lines>

 1  ROOT STORY: Node.js 7.5.0 on Darwin 64-bit, SB 3.0.0
  2017-02-23T09:01:53.344Z           storyboard INFO  Log filter: *:DEBUG

<a lot of empty lines>

 2  Run cmd yarn run build at packages/app

<a lot of empty lines>

 3  Run cmd yarn run build at packages/app-config

<a lot of empty lines>

 4  Run cmd yarn run build at packages/app-di

<a lot of empty lines>

 1  ROOT STORY: Node.js 7.5.0 on Darwin 64-bit, SB 3.0.0
  2017-02-23T09:01:53.344Z           storyboard INFO  Log filter: *:DEBUG

<a lot of empty lines>

 2  Run cmd yarn run build at packages/app

<a lot of empty lines>

 3  Run cmd yarn run build at packages/app-config

<a lot of empty lines>

 4  Run cmd yarn run build at packages/app-di

<a lot of empty lines>

 5  Run cmd yarn run build at packages/app-registry

<a lot of empty lines>

 1  ROOT STORY: Node.js 7.5.0 on Darwin 64-bit, SB 3.0.0
  2017-02-23T09:01:53.344Z           storyboard INFO  Log filter: *:DEBUG

<a lot of empty lines>

 2  Run cmd yarn run build at packages/app
  <ok>
  2017-02-23T09:01:55.124Z                 yarn INFO  Done in 1.35s.
  2017-02-23T09:01:55.142Z                 yarn INFO  
 3  Run cmd yarn run build at packages/app-config
  <ok>
  2017-02-23T09:01:54.597Z                 yarn INFO  Done in 0.80s.
  2017-02-23T09:01:54.618Z                 yarn INFO  


 5  Run cmd yarn run build at packages/app-registry
  <ok>
  2017-02-23T09:01:54.853Z                 yarn INFO  Done in 1.05s.
โœจ  Done in 2.23s.

NOTE: extracted issue from #13

oao bootstrap fails in parallel mode with "ENOENT: no such file or directory" for tarball in yarn cache

STR:

  1. Download this testcase: https://github.com/edmorley/neutrino-dev/archive/testcase-oao-parallel.zip
  2. Push to a new test repo that has Travis enabled

Expected:
The Travis run passes.

Actual:
The Travis run fails with errors like:

2017-10-10T16:16:07.815Z           storyboard WARN  โ”Œโ”€โ”€ REVEALED PAST LOGS BEGIN HERE (due to warning/error)
2017-10-10T16:16:02.162Z                 main TRACE โ”Œโ”€โ”€ Run cmd yarn install --frozen-lockfile at packages/neutrino-middleware-clean [CREATED]
2017-10-10T16:16:03.870Z                 yarn INFO  | yarn install v0.28.4
2017-10-10T16:16:05.368Z                 yarn INFO  | [1/4] Resolving packages...
2017-10-10T16:16:05.719Z                 yarn INFO  | [2/4] Fetching packages...
2017-10-10T16:16:07.816Z           storyboard WARN  โ””โ”€โ”€ REVEALED PAST LOGS END HERE
2017-10-10T16:16:07.815Z                 yarn ERROR | error An unexpected error occurred: "https://registry.yarnpkg.com/deepmerge/-/deepmerge-1.5.2.tgz: ENOENT: no such file or directory, open '/home/travis/.cache/yarn/v1/npm-deepmerge-1.5.2-10499d868844cdad4fee0842df8c7f6f0c95a753/.yarn-tarball.tgz'".

Examples:

This was using yarn 0.28.4 and oao 0.10.3.

Other observations:

  • I can't repro locally on: Windows 10 x64 + MSYS2 bash + yarn 0.28.4 + node 8.6.0 + oao 0.10.3.
  • Travis fails whether the yarn cache is empty or not

I believe this is due to yarnpkg/yarn#2629 - the summary of which is:

  • yarn's cache isn't currently safe for concurrent use
  • therefore callers of yarn who are doing concurrent things need to either:
    • use the --mutex option for locking (like lerna does after lerna/lerna#764)
    • manually handle cases where things go astray and fix up the cache/retry/... -> yuck

As such I think oao needs to either:

  • stop making parallel the default, since it can break the yarn cache
  • start using --mutex to avoid the concurrency issues. (I'm presuming this will slow things down slightly, but presumably not as much as disabling parallel mode entirely)

Pretty-print error output if a command failed in parallel mode

If you execute a command in a lot of your packages and it failed in one of them, the error output might not be completely visible which creates more effort for the developer since he has to cd into the failed package and re-run the command again to see the full output.

It would be cool if the whole output of the failed command would be printed after the summary.

Support installing deps with --frozen-lockfile

In CI it would be helpful to install package dependencies with a frozen lockfile so we can catch when package.json or lock files are out of step with what yarn installs.

Maybe a oao bootstrap --frozen flag, or proxy any unrecognized flag directly to yarn, e.g. yarn bootstrap --frozen-lockfile?

Let's talk

I am opening this discussion because, since I am using this very helpful library, there are many questions that come to my mind.

First of all, I use this client to help me building a monorepo with libraries based on Angular 2. Those libraries will by used by all Angular apps in my company. The architecture that I am trying to resolve is describes HERE.

As you can read in the README, there are a lot of issues that I can solve with oao but I still need to write some scripts for particular parts.

1st subject : please keep oao simple !

Like everybody, I started to use Lerna at the first time. But it was changing too much my way of working (I mean the company way). I am not saying it's a bad way but companies have their own rules and usually you can't change them easely.
When I tried to fit lerna to it, I had to add a lot of --skip-something options on each command...It was ugly and finally I don't felt that lerna could helps me.

First I wrote some scripts, then I tried Yerna...and finally oao....and I like it because it is simple and it use yarn (with good hacks ;-)) !

Oao helped me with oao bootstrapand oao all but for example, I cannot profite of oao prepublish/publish. Why ? Because It's already specifying a list of rules that I am forced to follow (some examples) :

  • oao prepublish:
    • Fields are copied from root package : Really, it's a very good idea but...why can't I choose the list of properties that I want to copy ?
    • ...
  • oao ppublish:
    • The current branch should be master : companies can have their own git workflow and they can use other branches. (don't react too fast like "Stupid guys, there is an option for that", continue reading ;-))
    • No uncommitted changes should remain in the working directory : during my build, I am modifying some dependencies version of each package.json to align versions of all of my libs (by example Angular version). I don't want to push those generated package.json, it's only when I am building the app for publishing...but I can't publish because they are modifications
    • Asks the user for an incremented master version : this is an example but other subjects are difficult to manage when we want that only a CI tool can publish.

I know I can solve a lot of those issue with options : --no-master, --no-confirm,...and probably may more coming like lerna. But why didn't we think in another way : putting in place a working core without rules (I am not saying removing checks like verifying that glob paths are directories, ...).
And later, adding options to add functionnalities.
It can also be a config file, for example :

oao.json

{
    "bootstrap": {
        "packages": [
            "packages/*",
            "other/*"
        ],
        "...": "..."
    },
    "prepublish": {
        "strictVersion": true,
        "copyFromRoot": {
            "packageFields": [
                "description",
                "author"
            ],
            "readme": true
        },
        "...": "..."
    },
    "publish": {
        "strictBranch": "master",
        "confirm": true,
        "strictCommit": true,
        "...": "..."
    }
}

Or we could even have a core that allows to plug a list of configurations, plugins, etc...

2nd subject : missing the build step !

Ref doc : Asks the user for confirmation that it has built all sub-packages for publishing (using something like yarn build).

Fine, when I build my app :

  • I create a dist/ folder at the root containing the list of built packages (I transpile and rollup my packages).
  • Next, I copy in the corresponding dist/, the package.json and I replace placeholders version for each package.
  • Next, I copy in the corresponding dist/, the README.md for each package.
  • ... many scripts for manual build

And now, you should think : "ok, but it's not my business". My issues here are :

  • If I want to publish, I need to publish each package of my dist folder and not the original ones...how do I do ?
  • If I want to work locally, I need to oao bootstrap my dist folder. Links will enter in conflict with the original ones...how do I do ?
  • ...

I think it could be good to have a relation between the source files and the built files. I mean, my tool should be able to take in account my dist/ folder at the root or in each package folder.
I don't know exactly how but, I think the build steps should be part of the tool.

3rd subject : Who cares ?

Me ;-) and many others. When I expose the subject during meetups, I feel that people are interested by the monorepos but they don't find a good tool, or they create many differents ways of doing it.

4th subject : Sorry

I you already know, english is not my main language so, in advance, sorry for my bad sentences probably too strict and not in a good and polite way

Final

Thank you if you read all of this. I am really open to your constructive criticism ! Today I am not enough good in node js developments to start those kind of project but I hope some of my ideas will became debates, issues, CR, etc...

publish enhancement

I would like to discuss some improvements:

  1. allow publish using master version, currently if master version (allSpecs[ROOT_PACKAGE].specs.version) is null and --new-version is specified then publish will be skipped.

I would like to improve publish command:

  • if --new-version is specified then it will be used;
  • if --new-version is not specified then master version will be used in case of missing git version;
  • if --new-version is not specified then git version will be used in case of successfully resolved next version;

Here is a simple fix for that:

// Get last tag and find packages requiring updates
  const lastTag = await gitLastTag();

  // Determine a suitable version number
  let nextVersion;

  if (!newVersion) {
    const masterVersion = await getMasterVersion(allSpecs, lastTag);

    if (masterVersion !== null && confirm) {
      nextVersion = await getNextVersion(masterVersion);
    } else {
      nextVersion = masterVersion;
    }
  } else {
    nextVersion = newVersion;
  }

  if (!nextVersion) {
    mainStory.error('No version have been specified');
    throw new Error('INVALID_VERSION');
  }

  const dirty = await findPackagesToUpdate(allSpecs, lastTag, nextVersion);
  if (!dirty.length) {
    mainStory.info('No sub-packages have been updated');
    return;
  }
  1. allow publish packages individually:
  • allow to use allSpecs[pkgName].version instead of master/git/--new-version versions;
  • skip publishing of already published packages, if [email protected] is published then it should be skipped.

oao all try to execute command on file in the packages directory

Hello,

I get this error when I try to execute a command using oao all :

$ oao all ls
2017-02-21T07:47:14.368Z           storyboard INFO  โ”Œโ”€โ”€ ROOT STORY: Node.js 7.5.0 on Win32 64-bit, SB 3.0.0 [CREATED]
2017-02-21T07:47:14.370Z           storyboard INFO  Log filter: *:DEBUG
2017-02-21T07:47:14.557Z                 main INFO  โ”Œโ”€โ”€ Run cmd ls at packages/lib-1 [CREATED]
2017-02-21T07:47:14.675Z                   ls INFO  | index.ts
2017-02-21T07:47:14.676Z                   ls INFO  | node_modules
2017-02-21T07:47:14.676Z                   ls INFO  | package.json
2017-02-21T07:47:14.681Z                   ls INFO  | rollup.config.js
2017-02-21T07:47:14.681Z                   ls INFO  | src
2017-02-21T07:47:14.682Z                   ls INFO  | yarn.lock
2017-02-21T07:47:14.683Z                   ls INFO  |
2017-02-21T07:47:14.688Z                 main INFO  โ””โ”€โ”€ Run cmd ls at packages/lib-1 [CLOSED]
2017-02-21T07:47:14.689Z                 main INFO  โ”Œโ”€โ”€ Run cmd ls at packages/lib-2 [CREATED]
2017-02-21T07:47:14.710Z                   ls INFO  | index.ts
2017-02-21T07:47:14.710Z                   ls INFO  | node_modules
2017-02-21T07:47:14.711Z                   ls INFO  | package.json
2017-02-21T07:47:14.711Z                   ls INFO  | rollup.config.js
2017-02-21T07:47:14.711Z                   ls INFO  | src
2017-02-21T07:47:14.711Z                   ls INFO  | yarn.lock
2017-02-21T07:47:14.715Z                   ls INFO  |
2017-02-21T07:47:14.716Z                 main INFO  โ””โ”€โ”€ Run cmd ls at packages/lib-2 [CLOSED]
2017-02-21T07:47:14.716Z                 main INFO  โ”Œโ”€โ”€ Run cmd ls at packages/lib-3 [CREATED]
2017-02-21T07:47:14.738Z                   ls INFO  | index.ts
2017-02-21T07:47:14.739Z                   ls INFO  | node_modules
2017-02-21T07:47:14.739Z                   ls INFO  | package.json
2017-02-21T07:47:14.739Z                   ls INFO  | rollup.config.js
2017-02-21T07:47:14.739Z                   ls INFO  | src
2017-02-21T07:47:14.739Z                   ls INFO  | yarn.lock
2017-02-21T07:47:14.742Z                   ls INFO  |
2017-02-21T07:47:14.744Z                 main INFO  โ””โ”€โ”€ Run cmd ls at packages/lib-3 [CLOSED]
2017-02-21T07:47:14.744Z                 main INFO  โ”Œโ”€โ”€ Run cmd ls at packages/tsconfig.json [CREATED]
cd: not a directory: packages/tsconfig.json
2017-02-21T07:47:14.765Z                   ls INFO  | node_modules
2017-02-21T07:47:14.766Z                   ls INFO  | package.json
2017-02-21T07:47:14.766Z                   ls INFO  | packages
2017-02-21T07:47:14.766Z                   ls INFO  | scripts
2017-02-21T07:47:14.767Z                   ls INFO  | yarn.lock
2017-02-21T07:47:14.770Z                   ls INFO  |
2017-02-21T07:47:14.771Z                 main INFO  โ””โ”€โ”€ Run cmd ls at packages/tsconfig.json [CLOSED]
2017-02-21T07:47:14.772Z           storyboard INFO  โ””โ”€โ”€ ROOT STORY: Node.js 7.5.0 on Win32 64-bit, SB 3.0.0 [CLOSED]

The last "block" is not a folder...

Bug: bootstrap removing subpackages dependencies declarations

I don't know why, but sometimes when I will run bootstrap the subpackages dependencies declarations are removed. I figure out that oao remove subpackages dependencies and them, put they again, I think that when some error happens in this process, the declarations are removed and aren't put again in package.json

--no-git during publish

Hello,

What do you think about having a --no-git during the publish process ? Or one option --no-[step] for each step related to a git action (like --no-master)

Git steps:

  • No uncommitted changes should remain in the working directory: on a CI environment, template files could be modified during the build and we don't want to commit these files because it's only neded for the pipeline or the publish
  • No unpulled changes should remain.: if the publish is done by a CI environment, a developer should always wait until the "build pipeline" to be finished.
  • Updates versions in package.json files, commits the updates, adds a tag and pushes all the changes.: the CI environment is not allowed to push.

Thx

Global options like `--src` should exist in package.json

Reason being, I have oao bootstrap --src '{examples,packages}/*' in my package.json, but if I do oao status or oao add, I have to redeclare this --src over & over again.

Having "oao": { "src": "{examples,packages}/*" } in package.json could help remedy this.

Declare babel-polyfill as dependency or peerDependency

Right now just doing an installation of oao and running it throws an error:

โฏ ./node_modules/.bin/oao bootstrap
module.js:471
    throw err;
    ^

Error: Cannot find module 'babel-polyfill'
    at Function.Module._resolveFilename (module.js:469:15)
    at Function.Module._load (module.js:417:25)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (/Users/eli/code/neutrino-dev/node_modules/oao/lib/index.js:4:1)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)

Looking at the code, it is indeed looking for babel-polyfill in the compiled code, but it's only listed as a dev dependency. This should be either a dependency or peer dependency.

publish : skip check updated package or force publish

Hello,

I am comming (again sorry :-p) with my continuous integration issues

After putting all skip options:

$ oao publish --no-master --no-confirm --no-check-uncommitted --no-check-unpulled --no-git-commit --version 0.1.0

Nothing happen during the publish:

2017-03-08T07:52:23.571Z           storyboard INFO  โ”Œโ”€โ”€ ROOT STORY: Node.js 7.6.0 on Win32 64-bit, SB 3.1.1 [CREATED]
2017-03-08T07:52:23.572Z           storyboard INFO  Log filter: *:DEBUG
0.7.3
2017-03-08T07:52:23.824Z           storyboard INFO  โ””โ”€โ”€ ROOT STORY: Node.js 7.6.0 on Win32 64-bit, SB 3.1.1 [CLOSED]
Done in 2.45s.

Is there a way to force the publish ?

watching

It would be nice to have ability to run watch tasks.

  "scripts": {
    "watch": "babel src --out-dir dist --source-maps --watch --copy-files"
  }

Currently oao freezes on the first package :(

Version upgrades and yarn.lock

There seems to be something out of step with bumping versions with dependent sub-packages. Let's take the following example:

package alpha:

{
  "name": "alpha",
  "version": "1.0.0"
}

package beta:

{
  "name": "beta",
  "version": "1.0.0",
  "dependencies": {
    "alpha": "1.0.0"
  }
}

If I want to make updates to alpha, I have to manually update the dependency version in beta to pick up the changes. If alpha has new dependencies, and thereby a new yarn.lock file, those deps won't be picked up in the yarn.lock file of beta.

Running bootstrap after this work doesn't make any changes either.

Feature: oao all with exceptions for some packages

Hello there! Firstly, thanks you for the oao tool, it's really awesome.

I've a feature request to introduce to oao all 'cmd' command a flag (may be --except <args...>) for excepting a list of packages for which to run the cmd command.

Example:

oao all 'webpack' --except pkg1 pkg3
packages
  - pkg1
  - pkg2
  - pkg3
  - pkg4

It'll run the webpack command for pkg2, pkg4, except for pkg1, pkg3.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.