eosio / demux-js-eos Goto Github PK

Currently, Updaters must re-implement contract logic to accumulate equivalent blockchain state. There should be some mechanism to capture this data directly.

If demux gets crashed in a block that is being forked it fails to restart. When we try to start demux it goes to an infinite loop trying to read the current and last block (my personal logs where I write the head and each block number read):

{"log":"1|demux    | Mongo Block State Header -- Block Num 17073622\n","stream":"stdout","time":"2018-09-18T14:11:26.729090615Z"}
{"log":"1|demux    | Mongo Getting blockNumber: 17073622\n","stream":"stdout","time":"2018-09-18T14:11:26.729468451Z"}
{"log":"1|demux    | Mongo Block State Header -- Block Num 17073623\n","stream":"stdout","time":"2018-09-18T14:11:27.229939692Z"}
{"log":"1|demux    | Mongo Getting blockNumber: 17073623\n","stream":"stdout","time":"2018-09-18T14:11:27.229959764Z"}
{"log":"1|demux    | Mongo Block State Header -- Block Num 17073624\n","stream":"stdout","time":"2018-09-18T14:11:27.728765716Z"}
{"log":"1|demux    | Mongo Getting blockNumber: 17073624\n","stream":"stdout","time":"2018-09-18T14:11:27.729217749Z"}
{"log":"1|demux    | Mongo Block State Header -- Block Num 17073625\n","stream":"stdout","time":"2018-09-18T14:11:28.230462666Z"}
{"log":"1|demux    | Mongo Getting blockNumber: 17073625\n","stream":"stdout","time":"2018-09-18T14:11:28.231375248Z"}

I suspect it is caused by the following code because this.blockHistory.length is zero when we are starting up demux.

// Continue if the new block is on the same chain as our history, or if we've just started
      if (expectedHash === actualHash || this.blockHistory.length === 0) {

I can't connect to mongo database. I always get this error, even if database has these collections. Also, I observed that if I create a new database adding say only transaction_traces, I'm getting error that this collection is missing. Here's my code and error:

I seem to hit issue with 1.8.1 PostGres. Any ideas how to fix this?

I meet this problem, my MongoDB need auth to login, and I create MongoActionReader by these:

 const actionReader = new MongoActionReader(
        "mongodb://USERNAME:PASSWORD@myDbIp:port/EOS",
        0,
        true,
        600,
        "EOS"
    );

and then i get this error:

 UnhandledPromiseRejectionWarning: Error: MongoActionReader must be initialized before fetching blocks.

I think the problem is the MongoActionReader can't be initial by my mongoEndpoint, can i get some help?

I am a developer of eos，See your comments about escorw on eosbet。I want to ask you about the excrow contract. Are you finished? Or have you found a replacement, if you have any, can you share it with me, thank you very much!

I keep running into this error after setting up using the example code in the repo. I've tried 3 different api endpoints. But all return same error.

The Mongo Action Reader does not find any actions if the producer_block_id is null. All my action traces come up with producer_block_id as null. I'm not sure why it comes up as null, could be something wrong with my setup. Either way, its preventing all of my handlers from executing because it will filter out all the actions.

We should definately put in a warning of some sort, since this caused me a couple hours tracing why my updaters weren't firing. And if theres not something wrong with my setup, probably will need a way to disable it.

I'm happy to open a PR for this but just wanna check if my setup is bugged first.

The code related to this is from demux-js-eos/src/MongoActionReader.ts in getBlock.

        const rawActions = await this.mongodb!.collection("action_traces")
          .find({
            block_num: blockNumber,
            producer_block_id: blockState.block_id,
          })
          .sort({ "receipt.global_sequence": 1 })
          .toArray()

Example Block

demux version: 3.1.3
demux-eos version: 3.0.1
eos-dev version: 1.5.2
eos cdt version: 1.4.1

In AbstractActionReader.getNextBlock(), this.getLastIrreversibleBlockNumber() is called before the initialized check, which causes an error to be immediately thrown in the MongoActionReader by this.throwIfNotInitialized()

The current MongoActionReader does not include inline actions in its action output, as there are currently issues with how inline actions are persisted via the MongoDB plugin. EOSIO/eos#5339 should address theses issues. After this is merged, we should be able to only require block_states and action_traces collections.

I would like to configure it to wait longer between retries... how would I go about doing that? I've been trying to figure out for days.

Hi! I am having trouble declaring a global variable in Demux which doesn't reset after effecting every EOS Transfer. Is there a way to declare this within the Updaters folder?

For example I want to get real time changes of this table, glad to accept every help

https://bloks.io/account/betdiceadmin?loadContract=true&tab=Tables&account=betdiceadmin&scope=betdiceadmin&limit=100&table=user

Using the MongoActionRead, sometimes, randomly, I receive this exception:

(node:48004) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'transactions' of undefined
    at MongoBlock.collectActionsFromBlock (/Users/leoribero/Workspace/phomo-news/services/backend/node_modules/demux-eos/dist/MongoBlock.js:14:49)
    at new MongoBlock (/Users/leoribero/Workspace/phomo-news/services/backend/node_modules/demux-eos/dist/MongoBlock.js:5:29)
    at MongoActionReader.<anonymous> (/Users/leoribero/Workspace/phomo-news/services/backend/node_modules/demux-eos/dist/MongoActionReader.js:54:27)
    at Generator.next (<anonymous>)
    at fulfilled (/Users/leoribero/Workspace/phomo-news/services/backend/node_modules/demux-eos/dist/MongoActionReader.js:4:58)
    at process._tickCallback (internal/process/next_tick.js:68:7)
(node:48004) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:48004) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

It only happens when demux is synced with the chain (which I believe it tries to read a next block number without this block being created)...

This is my demux instance declaration:

  const db = await MassiveProvider.getMassive()

  const actionHandler = new MassiveActionHandler(
    updaters,
    effects,
    db,
    dbConfig.schema,
  )
  // const actionReader = new NodeosActionReader(chainConfig.httpEndpoint, 1)
  const actionReader = new MongoActionReader(
    mongoDbConfig.host,
    50,
    false,
    600,
    mongoDbConfig.dbName,
  )

  await actionReader.initialize()

  const actionWatcher = new BaseActionWatcher(actionReader, actionHandler, 250)

  actionWatcher.watch()

when demux detect the fork block, any way to get the required infor?

error TS2705: An async function or method in ES5/ES3 requires the 'Promise' constructor. Make sure you have a declaration for the 'Promise' constructor or include 'ES2015' in your --lib option.

0|www | indexState:{ _id: 5c04dc8cb98dfadfa4dc9c20,
0|www | __v: 0,
0|www | blockHash: '01674e4e05ce9f0891bc9cbf12809dcc230560339de00caeaf0ad5e2dc6d63bd',
0|www | blockNumber: 23547470,
0|www | handlerVersionName: 'v1',
0|www | isReplay: false,
0|www | createTime: 2018-12-03T09:49:02.432Z,
0|www | updateTime: 2018-12-03T09:49:02.432Z }
0|www | blockHash:01674e4e05ce9f0891bc9cbf12809dcc230560339de00caeaf0ad5e2dc6d63bd blockNumber:23547470handlerVersionName:v1
0|www | You have triggered an unhandledRejection, you may have forgotten to catch a Promise rejection:

/root/.pm2/logs/www-error.log last 15 lines:
0|www | (node:33) DeprecationWarning: current URL string parser is deprecated, and will be removed in a future version. To use the new parser, pass option { useNewUrlParser: true } to MongoClient.connect.
0|www | (node:33) DeprecationWarning: collection.ensureIndex is deprecated. Use createIndexes instead.
0|www | Error: Cannot seek to block before configured startAtBlock.
0|www | at NodeosActionReader. (/root/demux_backend/node_modules/demux-eos/node_modules/demux/dist/AbstractActionReader.js:91:23)
0|www | at Generator.next ()
0|www | at /root/demux_backend/node_modules/demux-eos/node_modules/demux/dist/AbstractActionReader.js:7:71
0|www | at new Promise ()
0|www | at __awaiter (/root/demux_backend/node_modules/demux-eos/node_modules/demux/dist/AbstractActionReader.js:3:12)
0|www | at NodeosActionReader.seekToBlock (/root/demux_backend/node_modules/demux-eos/node_modules/demux/dist/AbstractActionReader.js:86:16)
0|www | at BaseActionWatcher. (/root/demux_backend/node_modules/demux/dist/BaseActionWatcher.js:62:45)
0|www | at Generator.next ()
0|www | at fulfilled (/root/demux_backend/node_modules/demux/dist/BaseActionWatcher.js:4:58)
0|www | at
0|www | at process._tickCallback (internal/process/next_tick.js:189:7)

I am experiencing an error while importing the demux-eos package following the instructions written in the readme.

What is not working

const {NodeosActionReader} = require("demux-eos/v1.8")

import {NodeosActionReader} from "demux-eos/v1.8"

Results in an import error.

Also in the doc sometimes the v for the version is missing like in NodeosActionReader setup section, last line of the text I attach here:

NodeosActionReader

Makes requests directly to a specified Nodeos API endpoint to obtain block data.
Setup

All that is required is a running Nodeos instance that has the chain_api_plugin enabled.
Example

const { BaseActionWatcher } = require("demux")
const { NodeosActionReader } = require("demux-eos/1.8")

What is working

const { NodeosActionReader } = require("demux-eos")

The retry function does not work as intended when the argument func returns a Promise.

func might be wrapped in a try-catch block but since it returns a Promise, any error thrown within it will not be propagated as expected:

try {
  return func() // Unwrapped promise, error will not be cought
} catch (err) {
  // ...
}

This results in uncaught network request errors for both MongoActionReader and NodeosActionReader, failing to retry as intended.

Since retry is an async function, a quick solution would be to unwrap the promise before returning it so that the try-catch can actually pick up thrown errors:

try {
  return await func()
} catch (err) {
  // ...
}