Git Product home page Git Product logo

gulp-multi-thread-task's People

Contributors

ct1994 avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

nhkyxx

gulp-multi-thread-task's Issues

handlers[message.type] is not a function

Any idea why these happens?

$ npx gulp lint:js2
[04:23:20] Using gulpfile ~/src/projectName/gulpfile.js
[04:23:20] Starting 'lint:js2'...
[04:23:20] spawning 4 worker
[04:23:22] Using gulpfile ~/src/projectName/gulpfile.js
[04:23:22] Starting 'lint:js2'...
[04:23:22] 'lint:js2' errored after 2 s
[04:23:22] TypeError: handlers[message.type] is not a function
    at Worker.<anonymous> (/home/bobby/src/projectName/node_modules/gulp-multi-thread-task/index.js:30:29)
    at Worker.emit (events.js:326:22)
    at Worker.EventEmitter.emit (domain.js:506:15)
    at ChildProcess.<anonymous> (internal/cluster/worker.js:32:12)
    at ChildProcess.emit (events.js:314:20)
    at ChildProcess.EventEmitter.emit (domain.js:506:15)
    at emit (internal/child_process.js:876:12)
    at processTicksAndRejections (internal/process/task_queues.js:85:21)
[04:23:22] Using gulpfile ~/src/projectName/gulpfile.js[04:23:22] Starting 'lint:js2'...
[04:23:22] The following tasks did not complete: lint:js2                                                                                                                 
[04:23:22] Did you forget to signal async completion?
[04:23:22] The following tasks did not complete: lint:js2
[04:23:22] Did you forget to signal async completion?
[04:23:23] Using gulpfile ~/src/projectName/gulpfile.js
[04:23:23] Starting 'lint:js2'...
[04:23:23] 'lint:js2' errored after 3.54 ms
[04:23:23] Error: write EPIPE
    at process.target._send (internal/child_process.js:806:20)
    at process.target.send (internal/child_process.js:677:19)
    at _requestFile (/home/bobby/src/projectName/node_modules/gulp-multi-thread-task/lib/message-handlers.js:75:13)
    at workerMessageHandlers (/home/bobby/src/projectName/node_modules/gulp-multi-thread-task/lib/message-handlers.js:110:3)
    at GulpMultiThreadTask (/home/bobby/src/projectName/node_modules/gulp-multi-thread-task/index.js:77:27)
    at /home/bobby/src/projectName/gulpfile.js:81:10
    at lint:js2 (/home/bobby/src/projectName/node_modules/undertaker/lib/set-task.js:13:15)
    at bound (domain.js:427:14)
    at runBound (domain.js:440:12)
    at asyncRunner (/home/bobby/src/projectName/node_modules/async-done/index.js:55:18)

I believe the write EPIPE errors were simply due to the gulp task ending prematurely due to the first handlers[message.type] is not a function error.

Here's the snippet of my gulp file:

const gulp = require('gulp')
const { GulpMultiThreadTask } = require('gulp-multi-thread-task')
const eslint = require('gulp-eslint7')
// ...

function lintJs2 (done, src) {
  return gulp.src(src)
    .pipe(eslint())
    .pipe(eslint.format('stylish'))
    .pipe(eslint.failAfterError())
}

gulp.task('lint:js2', function (done, src) {
  return GulpMultiThreadTask(
    'lint:js2',
    [
      './*.js',
      './{scripts,src}/**/*.js'
    ],
    lintJs2,
    {
      concurrency: Math.min(require('os').cpus().length / 2)
    }
  )
})

// ...

Plugin slightly slower performance compared to regular gulp

Hi,
I tried your plugin for my gulp task that processes images, but found that it was actually slower than without it.. Not sure if I'm doing something wrong, my task is not suitable for multi-threading or that the plugin might have some issue?

Situation
I have some directories with JPG, PNG and GIF image files. They have to be converted to WebP and placed besides their original counterpart.

System

  • PC: Windows 10 on quad core processor
  • Node: v14.8.0
  • Npm: v6.14.8

Directory structure

cdn/
    pictures/
         (around 120 .jpg, .png and .gif files, average filesize ~120kB)
    thumbs/
         (around 110 .jpg, .png and .gif files, average filesize ~10kB)

gulpfile.js

const { task, src, dest } = require('gulp');
const {GulpMultiThreadTask} = require('gulp-multi-thread-task');

const gulpimagemin = require('gulp-imagemin'); // For image handling
const imagemin_webp = require('imagemin-webp'); // Converts JPG/PNG to WebP
const imagemin_gif2webp = require('imagemin-gif2webp'); // Converts only GIF to WebP
const rename = require('gulp-rename');

function webpPicturesTask(done, srcGlob) {
  return src(srcGlob, { base: 'cdn/' })
    .pipe(gulpimagemin([
      imagemin_webp({
        lossless: false,
        method: 6,
      }),
      imagemin_gif2webp({
        lossy: false,
        mixed: false,
        minimize: false,
        method: 6,
        multiThreading: true,
      })
    ], {
      verbose: true
    }))
    .pipe(rename({ extname: '.webp' }))
    .pipe(dest('cdn/'))
  ;

task('webpPictures', function(done, srcGlob) {
  return GulpMultiThreadTask(
    'webpPictures',
    ['cdn/{pictures,thumbs}/**/*.{png,jpg,jpeg,gif}'],
    webpPicturesTask
  );
});

Results

  • Run as a normal task, it processed all 231 files in 54 seconds.
  • Run in a multi-threaded set-up as above, it did the same in 65 seconds. (Using 4 workers.)
    The additional time to spawn the workers at the start was very small (~1 sec), so I'm confused about the additional time it took.

Any idea what I'm doing wrong or why it is slower?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.