Git Product home page Git Product logo

Comments (19)

xiyuan-fengyu avatar xiyuan-fengyu commented on June 1, 2024 1

有一些功能和 webUI是相关的,没有关闭webUI的选项,但是你可以在@launcher初始化时,修改webUI的端口
image

from ppspider.

xiyuan-fengyu avatar xiyuan-fengyu commented on June 1, 2024 1

OnStart, OnTime, FromQueue 三种任务都是可以设置并行数和任务执行间隔的,默认并行数1,执行间隔1000ms,你需要把并行间隔参数exeInterval调小

from ppspider.

xiyuan-fengyu avatar xiyuan-fengyu commented on June 1, 2024 1

MongodbDao 和 NedbDao 都是 DbDao 的具体实现,仅提供了两者都已实现的统一操作;因为这原因,MongodbDao并没有提供额外的特有的操作实现;如果配置为mongodb的情况下,appInfo.db就是一个MongodbDao实例,你可以通过里面的dbPromise属性拿到mongodb的引用,进行mongodb的特殊操作;mongdb的说明文档可以在它的index.d.ts中找到方法说明
http://mongodb.github.io/node-mongodb-native/3.1/api/Collection.html#~updateWriteOpResult
image

from ppspider.

songbaiTalk avatar songbaiTalk commented on June 1, 2024

除了process.abort()外,框架本身是否有更优雅的方式在自定义的时机结束一个任务呢

from ppspider.

songbaiTalk avatar songbaiTalk commented on June 1, 2024

你的工具真心非常好用,能否加一下微信方便请教~,我的微信是13261539113,不方便的话也希望能留个邮箱之类的,非常感谢

from ppspider.

xiyuan-fengyu avatar xiyuan-fengyu commented on June 1, 2024

除了process.abort()外,框架本身是否有更优雅的方式在自定义的时机结束一个任务呢

界面上面有一个停止按钮,点击后可以停止整个系统,你可以发送一个信号来模拟这个请求,具体的看一下 WebUi的代码

邮箱 [email protected]

from ppspider.

xiyuan-fengyu avatar xiyuan-fengyu commented on June 1, 2024

其实可以在合适的时机调用appInfo.queueManager.setQueueRunning 来暂停所有的队列,不用停止整个应用

from ppspider.

songbaiTalk avatar songbaiTalk commented on June 1, 2024

wow,感谢如此即时的回复!

from ppspider.

songbaiTalk avatar songbaiTalk commented on June 1, 2024

当我用appInfo.queueManager.interrupteJob结束一个定时任务的时候,定时器还会继续执行并报错,我应该如何结束一个定时任务呢

from ppspider.

xiyuan-fengyu avatar xiyuan-fengyu commented on June 1, 2024

appInfo.queueManager.interrupteJob这个是用来中断一个正在执行的单个任务,不是用来停止一个队列的;如果要停止一个队列,用 appInfo.queueManager.setQueueRunning

from ppspider.

xiyuan-fengyu avatar xiyuan-fengyu commented on June 1, 2024

appInfo.queueManager.interrupteJob 这个方法是针对任务队列中的单个正在执行的任务,可以中断这个任务,但是这个队列并不会停止;你这里声明的是一个队列,队列中会周期性添加 url = 1, url =2, url = 3,三种子任务。从设计上来说,添加到这个队列中的子任务的行为应该是一致的,如果在运行过程中,希望部分子任务不运行,我想到两种方案,1:在OnTime回调函数中根据条件判定,希望停止的子任务,直接返回,不执行后面的逻辑;2:通过OnTime + AddtoQueue 周期性往其他队列中添加希望继续执行的子任务,将原本 OnTime回调函数中的任务逻辑转移到 FromQueue回调函数中

from ppspider.

songbaiTalk avatar songbaiTalk commented on June 1, 2024

了解~谢谢指教

from ppspider.

songbaiTalk avatar songbaiTalk commented on June 1, 2024

image
继续请教下,按你说的第二种方法,test Start会周期打印,但是testFromQueue中的内容不会周期执行,是使用姿势的问题吗

from ppspider.

xiyuan-fengyu avatar xiyuan-fengyu commented on June 1, 2024

AddToQueue 的参数中,filterType 默认为 BloonFilter,会根据job的key(如果没有设置,将使用url)进行去重,所以后面添加重复任务将会被过滤掉,在ui界面上可以搜索到这些被过滤掉的任务
正确的姿势如下

import {AddToQueue, FromQueue, Job, Launcher, NoFilter, OnTime} from "ppspider";

class TestTask {

    @AddToQueue({
        name: "test",
        filterType: NoFilter
    })
    @OnTime({
        urls: "",
        cron: "* * * * * *"
    })
    async onTime(job: Job) {
        console.log("test onTime");
        return ["0", "1", "2"];
    }

    @FromQueue({
        name: "test"
    })
    async fromQueue(job: Job) {
        console.log(job.url);
    }

}

@Launcher({
    workplace: "workplace",
    tasks: [
        TestTask
    ],
    workerFactorys: [
    ],
    webUiPort: 9001
})
class App {}

from ppspider.

songbaiTalk avatar songbaiTalk commented on June 1, 2024

好滴🙏🙏

from ppspider.

songbaiTalk avatar songbaiTalk commented on June 1, 2024

请问job的并行原理是怎样的,我发现我在定时任务的时候并行多个job的时候这些job也是轮流执行(比如每秒每个任务console一下,发现十个任务要10秒),大规模的时候没有达到并行的效率

from ppspider.

songbaiTalk avatar songbaiTalk commented on June 1, 2024

mongodb的操作是否支持upsert操作呢,另外update的返回结果中result的n字段是否为nMatched的含义?

from ppspider.

songbaiTalk avatar songbaiTalk commented on June 1, 2024

请问特殊操作是的方法具体应该是怎样呢,比如获取mongo索引,appInfo.db.dbPromise.indexes这样?

from ppspider.

xiyuan-fengyu avatar xiyuan-fengyu commented on June 1, 2024
const db = await appInfo.db.dbPromise;
// 或者  
appInfo.db.dbPromise.then(db => {

});

具体的mongodb操作就参考mongodb官方文档了

from ppspider.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.