Git Product home page Git Product logo

duck's Introduction

@cybozu/duck

Serves and builds an app with Google Closure Compiler/Library/Templates. An alternative to plovr.

npm version coverage

Install

$ npm i -D @cybozu/duck

Usage

duck <command>

Commands:
  duck serve [entryConfigDir]     Start dev server
  duck build [entryConfigDir]     Build Soy, deps.js and JS
  duck build:js [entryConfigDir]  Compile JS files
  duck build:soy                  Compile Soy templates
  duck build:deps                 Generate deps.js
  duck clean:soy                  Remove all compiled .soy.js
  duck clean:deps                 Remove generated deps.js
  duck completion                 Generate completion script for bash/zsh

Options:
  -v, --version  Show version number                                                     [boolean]
  -h, --help     Show help                                                               [boolean]

CLI options overwrite config file

duck serve

duck serve [entryConfigDir]

Start dev server

Options:
  --inputsRoot            A root directory to serve                                       [string]
  --closureLibraryDir     A root directory of Closure Library                             [string]
  --depsJs                A path to deps.js to save and load                              [string]
  --depsWorkers           The number of workers to analyze deps              [number] [default: 4]
  --config                A path to duck.config.js, the extension can be ommited          [string]
  --nonTTY, --noTTY, -n   Output in nonTTY mode                         [boolean] [default: false]
  --skipInitialBuild, -s  Skip initial building of Soy and deps.js      [boolean] [default: false]
  --port                  A port number to listen                         [number] [default: 9810]
  --host                  A host to listen                         [string] [default: "localhost"]
  -v, --version           Show version number                                            [boolean]
  -h, --help              Show help                                                      [boolean]

duck build

duck build [entryConfigDir]

Build Soy, deps.js and JS

Options:
  --entryConfigs, -e      Entry config files (this option ignores entryConfigDir)          [array]
  --closureLibraryDir     A root directory of Closure Library                             [string]
  --config                A path to duck.config.js, the extension can be ommited          [string]
  --concurrency, -c       Concurrency limit of Closure Compiler                           [number]
  --batch                 Build in batch mode (on AWS or local for debug)[choices: "aws", "local"]
  --reporters             Test reporters ("text", "xunit" or "json")   [array] [default: ["text"]]
  --reporterOptions       Test reporter options
  --printConfig, -p       Print effective configs for compilers         [boolean] [default: false]
  --depsJs                A path to deps.js to save and load                              [string]
  --nonTTY, --noTTY, -n   Output in nonTTY mode                         [boolean] [default: false]
  --depsWorkers           The number of workers to analyze deps              [number] [default: 4]
  --skipInitialBuild, -s  Skip initial building of Soy and deps.js      [boolean] [default: false]
  --soyJarPath            A path to Soy.jar                                               [string]
  --soyFileRoots          Root directories of soy files                                    [array]
  --soyClasspaths         Classpaths for Closure Templates                                 [array]
  --watch, -w             Re-compile incrementally when files change    [boolean] [default: false]
  -v, --version           Show version number                                            [boolean]
  -h, --help              Show help                                                      [boolean]

duck.config.js

Create a config file duck.config.js or duck.config.json on your project root. Set every path as a relative path from the location of duck.config.js.

module.exports = {
  /**
   * Common settings
   */
  // (Required) A path to Closure Library direcotry
  closureLibraryDir: "node_modules/google-closure-library",
  // (Required) A directory where entry config JSONs are stored flat
  entryConfigDir: "entry-configs",

  /**
   * Generating and loading deps.js
   */
  // A path to deps.js to save and load in build and serve commands
  depsJs: "build/deps.js",
  // Directories ignored when building deps.js
  depsJsIgnoreDirs: ["src/third_party"],
  // The number of workers to build deps.js. Node v10 uses processes and node v12+ uses threads. (default: 4)
  depsWorkers: 2,

  /**
   * Building Soy
   */
  // (Required) A path to Closure Templates JAR
  soyJarPath: "lib/closure-templates.jar",
  // (Required) Directories where Closure Templates .soy files are stored
  soyFileRoots: ["src/js"],
  // Classpaths for Closure Templates plugins
  soyClasspaths: ["lib/plugin.jar"],
  // Send srcs to closure-templates as relative paths from this.
  // Also change cwd to this and run closure-templates. (default: undefined)
  soySrcsRelativeFrom: "src",
  // Options for Closure Templates CLI
  soyOptions: {
    outputDirectory: "src/js/soy",
  },

  /**
   * Compiling JavaScript
   */
  // Concurrency of Closure Compiler (default: 1,000 if AWS batch mode, otherwise 1)
  concurrency: 4,
  // Build in batch mode with faast.js on "aws" for production or "local" for debug (default: undefined)
  batch: "aws",
  // Custom Closure Compiler package used in AWS batch mode (default: undefined)
  batchAwsCustomCompiler: {
    name: "my-custom-closure-compiler",
    version: "^1.0.0",
  },
  // Max chunk size in bytes for spliting return value from faastjs module (default: 204,800 (200kb))
  batchMaxChunkSize?: number;
  // Options for faast.js in batch mode. See https://faastjs.org/docs/api/faastjs.awsoptions
  batchOptions: {},
  // Reporters (choose from "json", "text" or "xunit")
  reporters: ["text", "xunit"],
  // Options for each test reporter
  reporterOptions: {
    text: {
      // Output errors to stderr or not
      stderr: false,
      // A directory where reporters output
      outputDir: "test-results/text",
    },
    xunit: {},
  },

  /**
   * Serve
   */
  // (Required) A root directory scanned to build deps.js and delivered as static assets
  inputsRoot: "src/inputs",
  // Hostname for serve command (default: 0.0.0.0)
  host: "localhost",
  // Port number for serve command (default: 9810)
  port: 1234,
  // Use HTTP/2 in serve command (deafult: false)
  http2: true,
  // Settings for HTTPS (HTTP/2) (default: not specified, HTTP is used)
  https: {
    // A path to a private key
    keyPath: "path/to/key.pem",
    // A path to a self-signed certificate
    certPath: "path/to/cert.pem",
  },
};

Also see examples.

Limitation

Tips

Batch mode using AWS Lambda

duck provides batch mode that compiles all entry points simultaneously in parallel on AWS Lambda with faast.js.

  1. Setting AWS credentials in Node.js (See AWS document)
  2. Configure batchOptions in duck.config.js. It's used for faast.js as AWSOptions.
const closureVersion = require("google-closure-compiler/package.json").version;

module.exports = {
  batchOptions: {
    region: "ap-northeast-1",
    awsLambdaOptions: {
      Runtime: "nodejs10.x",
    },
    include: ["path/to/your/source/**/*.js"],
    exclude: ["**/*_spec.js"],
    packageJson: {
      dependencies: {
        "google-closure-compiler-linux": closureVersion,
        "google-closure-library": closureVersion,
      },
    },
  },
};
  1. Run build or build:js command with --batch aws.
$ duck build --batch aws

How to debug in batch mode?

  • Use --batch local for local debugging
  • Use DEBUG=faast:info or other log level to get more debug information
  • Get logUrl from debug info and view it in CloudWatch logs
  • Use FAAST_PACKAGE_DIR=foo/bar to investigate a package sent to Lambda

Also see faast.js document for more information.

How to use HTTPS and HTTP2 in duck serve?

Create a self-signed certificate like

$ openssl req -x509 -newkey rsa:4096 -keyout duck-key.pem -out duck-cert.pem -days 365 -nodes -subj '/CN=localhost'

Then specify them and enable http2 in duck.config.js.

module.exports = {
  http2: true,
  https: {
    keyPath: "./path/to/duck-key.pem",
    certPath: "./path/to/duck-cert.pem",
  },
};

bash/zsh-completion for commands and options

Initial setting:

$ duck completion >> ~/.bashrc
# or
$ duck completion >> ~/.zshrc

Then, you can complete commands and options with TAB !

$ duck build:<TAB>
build:deps  -- Generate deps.js
build:js    -- Compile JS files
build:soy   -- Compile Soy templates

License

MIT License: Cybozu, Inc.

duck's People

Contributors

dependabot[bot] avatar gorohash avatar hikoma avatar itchyny avatar koba04 avatar nakajmg avatar pirosikick avatar renovate-bot avatar renovate[bot] avatar sakit0 avatar shisama avatar sosukesuzuki avatar teppeis avatar toshi-toma avatar zaki-yama avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

duck's Issues

Fail to install due to [email protected]

cause: google/closure-library#996

npm i -D @teppeis/duck
npm ERR! code ETARGET
npm ERR! notarget No matching version found for google-closure-compiler@^v20191027.0.1.
npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.
npm ERR! notarget 
npm ERR! notarget It was specified as a dependency of 'google-closure-deps'
npm ERR! notarget 

CI sometimes fails

I don't know why at the moment, but the Test workflow sometimes fails with the following error:

/home/runner/work/duck/duck/node_modules/workerpool/src/WorkerHandler.js:207
      me.worker.ready = true;
                      ^

TypeError: Cannot set property 'ready' of null
    at Worker.<anonymous> (/home/runner/work/duck/duck/node_modules/workerpool/src/WorkerHandler.js:207:23)
    at Worker.emit (events.js:314:20)
    at Worker.EventEmitter.emit (domain.js:483:12)
    at MessagePort.<anonymous> (internal/worker.js:201:53)
    at MessagePort.emit (events.js:314:20)
    at MessagePort.EventEmitter.emit (domain.js:483:12)
    at MessagePort.onmessage (internal/worker/io.js:80:8)
    at MessagePort.exports.emitMessage (internal/per_context/messageport.js:11:10)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] build:ci: `duck build --batch local -n`
npm ERR! Exit status 1

Update faastjs

When I updated the version of faastjs v5, our builds with AWS started failing, so I've reverted the change.
But it's not good to stick with v3, so we have to update the version to the latest.
(I'm not sure whether v4 works well or not)

The road to v1

I think it's the time that we release v1. What do you think?

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

  • chore(deps): update dependency @types/node to ~18.19.33
  • fix(deps): update dependency rimraf to ^5.0.7
  • fix(deps): update dependency semver to ^7.6.2
  • fix(deps): update dependency execa to v9
  • 🔐 Create all rate-limited PRs at once 🔐

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

Detected dependencies

github-actions
.github/workflows/test.yml
  • actions/checkout v4
  • actions/setup-node v4
  • codecov/codecov-action v4@54bcd8715eee62d40e33596ef5e8f0f48dbbccab
npm
package.json
  • @fastify/cors ^9.0.1
  • @fastify/static ^7.0.4
  • @teppeis/stream-to-observable ^0.3.1
  • chokidar ^3.6.0
  • common-tags ^1.8.2
  • execa ^8.0.1
  • faastjs ^8.0.75
  • fast-glob ^3.3.2
  • fastify ^4.26.2
  • google-closure-deps >=20210406.0.0
  • listr ^0.14.3
  • merge-options ^3.0.4
  • p-settle ^5.1.1
  • pino ^8.19.0
  • pino-pretty ^10.3.1
  • rimraf ^5.0.5
  • rxjs ^6.6.7
  • semver ^7.6.1
  • split2 ^4.2.0
  • strip-json-comments ^5.0.1
  • tempy 3.0.0
  • workerpool ^9.1.1
  • xmlbuilder ^15.1.1
  • yargs ^17.7.2
  • @cybozu/eslint-config ^22.0.2
  • @types/common-tags ^1.8.4
  • @types/listr ^0.14.9
  • @types/node ~18.19.32
  • @types/rimraf ^4.0.5
  • @types/semver ^7.5.8
  • @types/split2 ^4.2.3
  • @types/yargs ^17.0.32
  • @vitest/coverage-v8 ^1.3.1
  • eslint ^8.57.0
  • google-closure-compiler ^20231112.0.0
  • npm-run-all ^4.1.5
  • prettier ^3.2.5
  • standard-version ^9.3.2
  • typescript ^5.4.5
  • vitest ^1.3.1

  • Check this box to trigger a request for Renovate to run again on this repository

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

[feature] Specify dependent files in an entryconfig json.

In the current entryconfig json format, only directories are allowed to be written in paths in an entryconfig json file.

However, I would like to specify dependent js files explicitly so that I can find all dependent files in the config file and, moreover, I can know if there are some unexpected dependencies without changing the current source code structure.

webpack throws an error with Node.js v18

faastjs v3 depends on webpack v4 that does not support Node.js v17+

[email protected]+ supports Node.js v17+.
To use [email protected]+, we need to update faastjs to v5.4.2+ (faastjs/faast.js@ef9968a)


Error detail:

See https://github.com/cybozu/duck/runs/7623057363?check_suite_focus=true#step:6:40

> duck build --batch local -n

[01:30:32] Compile Soy templates [started]
[01:30:32] Compile Soy templates [skipped]
[01:30:32] Generate deps.js [started]
[01:30:32] → Analyzing dependencies
[01:30:32] → Generating deps.js
[01:30:32] → Generated: /home/runner/work/duck/duck/examples/build-and-serve/build/deps.js
[01:30:32] Generate deps.js [completed]
[01:30:32] Compile JS files [started]
[01:30:33] → Initializing batch mode
Error: error:0308010C:digital envelope routines::unsupported
    at new Hash (node:internal/crypto/hash:71:19)
    at Object.createHash (node:crypto:133:10)
    at module.exports (/home/runner/work/duck/duck/node_modules/webpack/lib/util/createHash.js:135:53)
    at NormalModule._initBuildHash (/home/runner/work/duck/duck/node_modules/webpack/lib/NormalModule.js:417:16)
    at handleParseError (/home/runner/work/duck/duck/node_modules/webpack/lib/NormalModule.js:471:10)
    at /home/runner/work/duck/duck/node_modules/webpack/lib/NormalModule.js:503:5
    at /home/runner/work/duck/duck/node_modules/webpack/lib/NormalModule.js:358:12
    at /home/runner/work/duck/duck/node_modules/loader-runner/lib/LoaderRunner.js:373:3
    at iterateNormalLoaders (/home/runner/work/duck/duck/node_modules/loader-runner/lib/LoaderRunner.js:214:10)
    at iterateNormalLoaders (/home/runner/work/duck/duck/node_modules/loader-runner/lib/LoaderRunner.js:221:10)

[feature] HTTPS

Run duck serve in HTTPS to prevent mixed content.
In addition, HTTP/2 may improve performance.

Error with `soyOptions.outputDirectory` for deleting a soy file

Closure Templates requires either soyOptions.outputDirectory or soyOptions.outputPathFormat.
clean:soy and build:soy --watch commands work with outputPathFormat but throw errors for deleting soy files with outputDirectory option.

$ duck clean:soy -n
[18:03:17] Clean up soy.js [started]
[18:03:17] Clean up soy.js [failed]
[18:03:17] → Cannot read properties of undefined (reading 'replace')
TypeError: Cannot read properties of undefined (reading 'replace')
    at Object.cleanSoy (node_modules/@cybozu/duck/lib/commands/cleanSoy.js:17:10)
    at node_modules/@cybozu/duck/lib/cli.js:301:45
    at processTicksAndRejections (node:internal/process/task_queues:96:5) {
  context: [Object: null prototype] {}
}

Update TypeScript to v4.6+

tsc throws errors with [email protected].

$ npm run build       

> @cybozu/[email protected] build
> tsc

node_modules/@types/verror/index.d.ts:31:5 - error TS2416: Property 'cause' in type 'VError' is not assignable to the same property in base type 'Error'.
  Property 'message' is missing in type '() => Error | undefined' but required in type 'Error'.

31     cause(): Error | undefined;
       ~~~~~

  node_modules/typescript/lib/lib.es5.d.ts:1023:5
    1023     message: string;
             ~~~~~~~
    'message' is declared here.

node_modules/@types/verror/index.d.ts:31:5 - error TS2425: Class 'Error' defines instance member property 'cause', but extended class 'VError' defines it as instance member function.

31     cause(): Error | undefined;
       ~~~~~


Found 2 errors in the same file, starting at: node_modules/@types/verror/index.d.ts:31

It will be resolved in next TypeScript 4.8.

See

Ignore compiler warnings

Currently, duck build:js ignores warnings of compiling.
But we have a case that we want to get the warnings.
I have two options for this.

  1. Prints warnings to stderr using console.error koba04@622e593
  2. Add --strict option that treats any warnings as errors koba04@59238d9

We could combine the options, which means that if --strict option is disabled, warnings are printed to stderr.

@teppeis Do you have any ideas for this issue?

Run lint on GitHub Actions

Currently, we only run the test script on GitHub Actions, so I'd like to run the lint script as well.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.