Git Product home page Git Product logo

makit's Introduction

makit

npm version downloads Build Status Coveralls dependencies semantic-release GitHub issues David David Dev DUB license Commitizen friendly

Purposes and Principles:

  • Minimal Concepts. It's intended to be a general purpose build automation tool just like GNU Make. Do not introduce unnecessary concept to keep it simple and stupid.
  • Less Restrictions. It should be as open as GNU Make, makit doesn't expect recipes return anything, doesn't even requrie a recipe for its rule definition, and doesn't care about the actual output of recipes.
  • JavaScript Style. Recipes can be written as callback style, Promise style or just synchronous style. Automatic variables are replaced by camelCased equivalents. Wildcard in static patterns are replaced by JavaScript RegExp and globs.

API Spec: https://searchfe.github.io/makit/modules/_index_.html

Get Started

Basically, the syntax is as simple as Makefiles but with a .js extension. A Makefile.js contains a set of rules, each of which consists of 3 parts:

  • target: either a filepath string, a glob string, or a RegExp object.
  • prerequisites: list of filepath strings, functions that return a list of strings, or list of strings and functions. functions here can be either sync or async.
  • recipe: an optional function, can be either sync or async.

Suppose we have a makefile.js containing the following contents:

const { rule } = require('makit')

rule('all', ['a1.min.js'])  // default rule

rule('a1.min.js', ['a.js'], function () {
    const src = readFileSync(this.dependencies[0], 'utf8')
    const dst = UglifyJS.minify(src).code
    writeFileSync(this.target, dst)
})

When we run makit(which is equivelant to make all cause all is the first rule), makit tries to make the target all which requires a1.min.js so the second rule will be applied firstly and its recipe is called to generate the target a1.min.js. The prerequisites for all has been fully resolved now and makit then tries to call its recipe, which is not defined for the above case so makit will just skip call the recipe and assume the target all is made successfully.

See /demo directory for a working demo. For more details see the typedoc for .rule()

Config

The makit CLI supports --help to print usage info:

makit.js [OPTION] <TARGET>...

Options:
  --version       Show version number                             [boolean]
  --makefile, -m  makefile path                                    [string]
  --database, -d  database file, will be used for cache invalidation
                                                   [default: "./.makit.db"]
  --require, -r   require a module before loading makefile.js or
                  makefile.ts                         [array] [default: []]
  --verbose, -v   set loglevel to verbose                         [boolean]
  --debug, -v     set loglevel to debug                           [boolean]
  --loglevel, -l  error, warning, info, verbose, debug
                                                   [choices: 0, 1, 2, 3, 4]
  --graph, -g     output dependency graph        [boolean] [default: false]
  --help          Show help                                       [boolean]

Or specify in package.json:

{
  "name": "your package name",
  "dependencies": {},
  "makit": {
    "loglevel": 2,
    "makefile": "makefile.ts",
    "require": ["ts-node/register"]
  }
}

Async (Promise & Callbacks)

When the recipe returns a Promise, that promise will be awaited.

rule('a1.min.js', ['a.js'], async function () {
    const src = await readFile(this.dependencies[0], 'utf8')
    const dst = UglifyJS.minify(src).code
    await writeFile(this.target, dst)
})
// equivelent to
rule('a1.min.js', ['a.js'], async ctx => {
    const src = await readFile(ctx.dependencies[0], 'utf8')
    const dst = UglifyJS.minify(src).code
    await writeFile(ctx.target, dst)
})

Callback style functions also work:

rule('clean', [], (ctx, done) => rimraf('{*.md5,*.min.js}', done))

Dynamic Dependencies

// `makit a.js.md5` will make a.js.md5 from a.js
rule('*.js.md5', ctx => ctx.target.replace('.md5', ''), async function () {
    const src = await this.readDependency(0)
    await this.writeTarget(md5(src))
})

Similarly, async prerequisites functions, i.e. functions of return type Promise<string> or Promise<string[]>, are also supported.

Matching Groups and Reference

Makit uses extglob to match target names. Furthermore it's extended to support match groups which can be referenced in prerequisites.

// `makit output/app/app.js` will make app.js.md5 from a.ts
rule('(output/**)/(*).js', '$1/$2.ts', async function () {
    return this.writeTarget(tsc(await this.readDependency()))
})
make('output/app/app.js')

Dynamic Prerequisites

It's sometimes handy to call make() within the recipe, but global make() is not valid in recipes. For example the following rule is NOT valid:

const { rule, make } = require('makit')

rule('bundle.js', 'a.js', async (ctx) => {
    await make('b.js')
    const js = (await ctx.readFile('a.js')) + (await ctx.readFile('b.js'))
    ctx.writeTarget(js)
})

We introduce rude() to facilitate this situation, a ctx.make API is available inside the recipe of rude:

const { rude } = require('makit')

rude(
    'bundle.js', [], async ctx => {
        await ctx.make('a.js')
        await ctx.make('b.js')
        const js = (await ctx.readFile('a.js')) + (await ctx.readFile('b.js'))
        return ctx.writeTarget(js)
    }
)

A pseudo rule with bundle.js.rude.dep as target, the contents of which is the actual dependencies, will be generated for each rude().

makit's People

Contributors

harttle avatar semantic-release-bot avatar qiansc avatar dependabot[bot] avatar

Stargazers

Andy Chen avatar bys avatar ShiJie avatar lingxiaoguang avatar  avatar shenzhou avatar 张启超 avatar PENG XING(Yongdi Yu) avatar 熊文杰 avatar Jenkey2011 avatar  avatar

Watchers

James Cloos avatar  avatar corcordium avatar

makit's Issues

基于 stats.mtimeMs 不够稳定

当两个任务的执行时间很接近时,可能会存在 mtime 相同的 target 和 prerequisites。这时文件变化检查就会失效。

现状

mtime 检查采用 dmtime >= mtime ,只要 target 不比 prerequisites 修改更晚就需要重新执行 recipe。这里的假设是:如果一个 target 根它的依赖创建时间非常接近,那么创建 target 的成本就很小,因此不妨重新创建一次。

问题

  • 当 target 被其他目标依赖的情况,这时 target 的所有祖先节点都会被重新构建。如果 target 是一个公共库那么影响会很大,因为几乎所有(依赖于这个公共库的)目标都失效了。

Daniel J. Bernstein 在 redo build system 中还提到 make 工作方式的另外几个问题:

  • The file isn't correct while it's being rebuilt; it's truncated. This is a problem for target files that are used by other programs.
  • If the build process is suddenly killed (for example, by a power outage), the truncated file is left in place. This is a problem for all target files: make sees the truncated file and assumes, incorrectly, that the file is up to date.

方案

计划在每个 recipe 完成之前 wait 3ms 来缓解问题,但完全解决需要一个类似数据库的东西存在本地来替代 mtime(可以 fallback 到 mtime)。

rule 的 target 参数不支持部分语法

  • rule('**/!(*.module).ts', 'deps', ctx => {}); : 这里的否定查找目前不支持
  • rule('**/(tsconfig.json)', '/$1', ctx => {});: 分组里只有一项匹配时, 无法匹配

默认不输出make状态

目前根 make 的行为一致,默认会输出每次 make 时的 target、以及 parent、parent 的 parent,…… 但考虑 JavaScript 社区的特殊性:

  • 异步 I/O。多个规则同时在跑,不可避免会有乱序日志,其实不可读。
  • 依赖关系比较明显。很多规则是类似 index.min.js 依赖于 index.js,打印出来的信息没太多用。

需要把它默认为不输出,通过 verbose 参数来开启。

静态模式支持

make 静态模式可以简化很多重复代码,例如:

// 现状
rule('*.min.js', ctx => ctx.getTargetPath().replace(/\.min\.js$/, '.js'), recipe)
// 支持静态模式后
rule('*.min.js', '%.min.js: %.js', recipe)

环状依赖检查性能优化

目前在 make 中调用 DirectedGraph#checkCyclic() 来检查是否存在环。每次代价是 O(n),总体 O(n^2),n 为 make 被调用的次数。

优化思路:增量构造一个 Set 穿透每个 make dfs 中从 根 到 叶 的路径。每次代价减小到 O(1)。

prerequisites 解决无法按顺序进行

目前 makit 中 prerequisites 的解决是无序的(Promise.all()),而 GNU make 是从左到右按顺序的。我们会有这种场景不能实现:

rule('dev', ['build', 'deploy'])

考虑如果不影响性能的话(如果target足够多即使每条顺序执行,并发应该也够)可以按顺序来。否则需要引入其他机制来提供按顺序的功能。

读取局部安装的 makit

全局安装的 makit 读取局部安装的 makit 并替换掉自己。这样总是可以全局安装一个,并且遵守本地安装的版本。可以用 child_process 的 exec 来实现。具体俩功能:

  1. 全局 makit 尝试调用本地 makit
  2. 本地 makit 不存在时提示(参考 mocha、eslint)

并发数限制

目前存在的问题是任务太多时,task 队列中 push 了太多任务导致 ctrl-c 无法及时响应。需要限制并发执行的任务,但要考虑依赖关系,避免死锁的情况。

shell 自动补全 target

对于能匹配目标的已经存在的文件,固定的目标字符串,都可以进行补全。

getPathToRoot() 不正确

循环依赖检测抛出异常时,路径不正确。可能是获取路径是不正确,也可能是循环依赖检测本身工作不正常。

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.