google / mono_repo.dart Goto Github PK
View Code? Open in Web Editor NEWAllows easy management of repositories with multiple Dart packages
Home Page: https://pub.dev/packages/mono_repo
License: BSD 3-Clause "New" or "Revised" License
Allows easy management of repositories with multiple Dart packages
Home Page: https://pub.dev/packages/mono_repo
License: BSD 3-Clause "New" or "Revised" License
I'd like to run pub global run mono_repo pub run build_runner build
to trigger build for all of the packages under the repository.
In many mono_repo setups we have a sibling packages that depends on each others.
It would be neat there was a command to generate a dependency_overrides
for dependencies that are siblings packages.
I'm not sure how this would work, we'll probably need a command to add/remove the dependency overwrites, unless pub ignores them when publishing packages.
The generated shell script tool/travis.sh
from the travis command is using a weird encoding causing some of our tools to treat it as binary instead of text.
If you provide the ordering of stages in the top level mono_repo.yaml it is not preserved in the output .travis.yml
file.
The top level config is mono_repo.yaml
The per-package config is .mono_repo.yml
These are inconsistent with their usage of a leading .
and with the file extension. Can we align them?
It would be nice if we could specify certain tasks that should be combined into a single travis job. This would be useful for tasks that are dominated by startup time like dartfmt, and would help speed up presubmits.
Add the ability to override a command on a specific subfolder in the mono_repo.yaml
Potentially we'd have
// mono_repo.yaml
pkg_a:
published: true
pkg_b:
published: true
packages_get: flutter pakages get
packages_update: flutter packages upgrade
we have a mono_repo with both flutter, web, and pure dart packages. when we run "mono_repo check" we get this error:
mono_repo check
Unhandled exception:
Invalid argument (data): No clue how to deal with `{sdk: flutter}`.: Instance of 'YamlMap'
Right now the stages section of the top level mono_repo.yaml file is required to be a map where you have to provide the keys name
and if
.
I had to write:
travis:
stages:
- name: analyze_and_format
if: 'true'
- name: unit_test
if: 'true'
- name: build
if: 'true'
When all I want is:
travis:
stages:
- analyze_and_format
- unit_test
- build
If there are other packages defined, those should be ran!
dart:
- dev
- stable
stages:
- sample1: "valid dart task string name"
- sample2:
- "valid dart task string name"
- script: "one line script invocation"
- name: foo
script:
- "script1"
- "script2"
- dartanalyzer: --hints-as-warnings .
- name: analyze_fun
dartanalyzer:
- name: analyze_fun2
dartanalyzer: --hint-as-warnings .
- name: analyze_fun3
dartanalyzer:
- analyze # ERROR! must have at least one action!
- test #ERROR! not allowed!
- sample1 #ERROR! duplicate!
- first_in_another_.monorepo.yml #ERROR! ordering cannot contradict across...things
exclude:
- dart: stable
jobs:
- foo
- bar # ERROR! doesn't exist as a name above!
Most of the packages in https://github.com/dart-lang/build want the exact same travis config (actually, all of them I think), but today they all need their own .travis.yml file.
It would be nice if I could provide like a default.travis.yml in the root, and it would apply that to any package which didn't have one.
In angular
, we declare a few cache directories in our travis config. We should be able to define these in child configs and copy them to the main one as part of the mono_repo travis
command.
I need this today:
# MANUALLY ADDED. Do not remove.
# Gives more resources on Travis (8GB Ram, 2 CPUs).
# Do not remove without verifying w/ Travis.
sudo: required
addons:
chrome: stable
# END MANUAL EDITS.
For mono repositories that have nested packages it would be nice to run commands in recursive mode by default.
What I want is to set the version for a specific package in the base repo. If the version changes all submodules using the this package should update their version accordingly. Should work for "path" also.
I realize this is not directly related to being a "mono repo", but I'm more and more seeing this package becoming something that is just useful for developing packages, not only mono repositories (we need a cooler name, too).
I'd love to spot-check packages that rely on mine before I publish. Maybe as easy as a mechanism for either locally (or on travis) running the tests/presubmit for another package with the versions overridden to the master of my package.
We already have a tool/travis.sh
script, it just needs to be invoked with all the configurations of the PKG and TASK environment variables.
The file is missing the import 'dart:async';
directive.
This means that pub global activate mono_repo
currently exits with
Failed to precompile mono_repo:mono_repo:
file:///Users/philippschiffmann/.pub-cache/hosted/pub.dartlang.org/mono_repo-1.2.0/lib/src/runner.dart:36:3: Error: 'Future' expects 0 type arguments.
Future<Null> runCommand(ArgResults topLevelResults) async {
^
file:///Users/philippschiffmann/.pub-cache/hosted/pub.dartlang.org/mono_repo-1.2.0/lib/src/runner.dart:36:16: Error: The return type of the method 'MonoRepoRunner::runCommand' is invalid-type, which does not match the return type of the overridden method (dart.async::Future<dart.core::Null>).
Change to a subtype of dart.async::Future<dart.core::Null>.
Future<Null> runCommand(ArgResults topLevelResults) async {
^
on OS X with Dart Version 2.0.0.
I think mono_repo would need a custom solution for this because of the nested packages etc, for any PR build it should check what packages had changes, and fail if the corresponding CHANGELOG was not updated.
It should also be overridable - probably with a custom label on the PR or specific message in the description.
It's possible some tasks could modify the local checkout – would be good to offer a check before each task is run to make sure it's clean
At least it appears that way, when I run it on the https://github.com/dart-lang/build repo it ends up changing the .travis.yml but not in a meaningful way (it just re-orders things).
It looks like it used to be sorted alphabetically but isn't any longer, but maybe on mac it just happened to work that way and on linux its different.
... across different sub-packages. No use if pkg_a uses 0.1.0
and pkg_b uses 0.2.0
.
... basically, something that is usable to help maintain packages/repos, not just for mono repositories. See dart-lang/mockito#100 (review) for some context.
As you mono_repo grows in size, the number of travis jobs tends to increase dramatically. It would be nice if we had a way of grouping tasks across packages to run together in the same travis job.
For instance see a recent build on the build repo, https://travis-ci.org/dart-lang/build/builds/512125780?utm_source=github_status&utm_medium=notification. We now have 25 different analyze/format tasks, which spend the majority of their time in setup and not actually doing work. These should ideally be groupable into a single travis job (as an opt in).
Add a group_stages
(or maybe group_by
?) key to the travis
section of the mono_repo.yaml
. This will accept a list of stage names.
We will create a single travis job (per sdk version) for each of those named stages.
This would reduce the number of travis jobs for the analyze_and_format
stage in the build mono_repo to 4, from 25 (we test on 4 different sdks right now).
Prerequisites:
Install mono_repo, according to README.md
> pub global activate mono_repo
git repo with folder structure
drwxr-xr-x 11 eugene.kot admin 374 May 28 18:30 .
drwxr-xr-x 9 eugene.kot admin 306 May 25 01:11 ..
drwxr-xr-x 15 eugene.kot admin 510 May 28 18:54 .git
-rw-r--r-- 1 eugene.kot admin 17 May 28 17:42 .gitignore
drwxr-xr-x 9 eugene.kot admin 306 May 28 18:49 .idea
-rw-r--r-- 1 eugene.kot admin 1067 May 22 00:29 LICENSE
-rw-r--r-- 1 eugene.kot admin 2134 May 28 17:42 README.md
drwxr-xr-x 16 eugene.kot admin 544 May 28 17:42 mobile
drwxr-xr-x 11 eugene.kot admin 374 May 28 17:42 models
drwxr-xr-x 11 eugene.kot admin 374 May 28 17:42 server
drwxr-xr-x 12 eugene.kot admin 408 May 28 18:50 web
Subfolders are dart packages. And it's really annoying to run pub get
in all of them. I thought that this tool can help me with that, so i ran
$ pub run mono_repo pub get
> Could not find a file named "pubspec.yaml" in "/Users/eugene.kot/reps/test".
Okay, maybe i should init it first? 🤔
$ pub run mono_repo pub init
> Could not find a file named "pubspec.yaml" in "/Users/eugene.kot/reps/test".
Maybe try inside of any repo
$ cd web
$ pub run mono_repo init
> Found `pubspec.yaml` in the current directory. Not supported.
So how to use it? And for what?
Thank you!
For angular, we need to have custom commands for running tests (as opposed to just pub run test
). Right now, there is no way to specify this in .travis.yml
files.
One alternative is to do what we're doing now, which is to have custom tool/travis.sh
for sub-packages.
At the end of each failed task – and maybe a summary at the end?
CC @jakemac53
For instance if you have:
dart_task:
# Run the presubmit tests as well (usually skipped).
- test: --run-skipped
then in the tool/travis.sh
you will get:
pub run test --run-skipped
but it will do that for all packages, not just the one package which had it added
I had to hack the produced travis.yml file to install chrome for the build package, it would be good if we could enable this generally somehow... ideally with static config so you don't have to remember to pass the command line option each time.
We should have a command for publishing all of the packages, based on a specified order.
Ideally, it would also remove the dependency overrides while publishing.
Is there an option to generate 'pub get' instead of 'pub upgrade' in travis.sh?
Travis appears to schedule tasks in roughly the order they appear in the matrix - it would be nice if we could change the ordering of the things in the env
section so that we can start the slower tasks first.
One way this could be done is by allowing users to specify an ordering by task name and/or package name?
We added a dependency_override that contained a reference to github:
build:
git:
url: https://github.com/dart-lang/build.git
path: build
running mono_repo check
had the following error:
Unhandled exception:
Invalid argument (data): No clue how to deal with `{git: {url: https://github.com/dart-lang/build.git, path: build_runner}}`.: Instance of 'YamlMap'
#0 new DependencyData (package:mono_repo/src/pubspec.dart:41)
#1 new Dependency.parse (package:mono_repo/src/pubspec.dart:17)
#2 Pubspec._getDependencies.<anonymous closure> (package:mono_repo/src/pubspec.dart:109)
#3 YamlNode&MapMixin.forEach (dart:collection/maps.dart:52)
#4 Pubspec._getDependencies (package:mono_repo/src/pubspec.dart:108)
#5 new Pubspec.fromJson (package:mono_repo/src/pubspec.dart:99)
#6 check.<anonymous closure> (package:mono_repo/src/check_command.dart:38)
#7 _HashVMBase&MapMixin&&_LinkedHashMapMixin.forEach (dart:collection-patch/dart:collection/compact_hash.dart:360)
#8 check (package:mono_repo/src/check_command.dart:33)
<asynchronous suspension>
#9 CheckCommand.run (package:mono_repo/src/check_command.dart:26)
#10 CommandRunner.runCommand (package:args/command_runner.dart:194)
<asynchronous suspension>
#11 CommandRunner.run.<anonymous closure> (package:args/command_runner.dart:109)
#12 new Future.sync (dart:async/future.dart:222)
#13 CommandRunner.run (package:args/command_runner.dart:109)
#14 main (file:///Users/alorenzen/.pub-cache/hosted/pub.dartlang.org/mono_repo-0.1.0/bin/mono_repo.dart:19)
<asynchronous suspension>
#15 _startIsolate.<anonymous closure> (dart:isolate-patch/dart:isolate/isolate_patch.dart:275)
#16 _RawReceivePortImpl._handleMessage (dart:isolate-patch/dart:isolate/isolate_patch.dart:163)
Since the individual jobs use env
it's not safe to have an environment configured at the top level without using global
. This is an easy mistake to make so we should detect that it's happening and warn.
So folks can do pkg/foo
for the foo
package, etc
I need this today:
cache:
directories:
- $HOME/.pub-cache
# MANUALLY ADDED. Do not remove.
- _benchmarks/.dart_tool
- _goldens/.dart_tool
- angular_router/.dart_tool
- angular_test/.dart_tool
# END MANUAL EDITS.
The logic would be essentially this:
As @jakemac53 said in #66 (review)
nit: might as well fix this to do p.join(p.dirname(Platform.resolvedExecutable), 'pub')
When working with multiple packages its a pain to keep files in sync. For example you often want the same analysis_options.yaml across packages. It would be great to have a consistent way of doing this.
I just confused myself when I ran mono_repo presubmit
and got the same error printed to the file for all actions "PKG environment variable must be set!".
I finally just ran mono_repo travis
to regenerate the script and this fixed everything. Maybe the command should detect a version mismatch with the script and do something smarter.
... to at least sanity check the version range.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.