Git Product home page Git Product logo

Comments (1)

Groxan avatar Groxan commented on August 18, 2024

Hi! That error means that responses from the node RPC (in particular, /chains/main/blocks/{level}) don't contain necessary data (e.g. metadata fields). You are not the first one who asked this question, so I will write a detailed explanation and pin this issue :)

Why blocks may not contain metadata fields

Roughly speaking, there are two types of storage in the node:

  • block storage - contains all blocks, that consists of a block header + list of operations;
  • context storage - contains additional data computed for each block, like balance updates, bigmap diffs, baking rights, etc.

So, when you do RPC request /chains/main/blocks/{level} the node looks for that block in the block storage and then looks for the context for that block in the context storage. If there is no context for that block, the node returns only raw block data, which is not enought for the indexer.

Why there may be no context for some blocks

When the node is working and receiving new blocks, it saves the blocks and also computes and saves context for those blocks. For example, if you check the context for the head block (/chains/main/blocks/head/metadata) it will always be available.

However, it may be missed for old blocks due to two possible reasons:

  • you are using the "rolling" mode, which prunes the context older than 5 cycles (~15 days);
  • you have just imported a fresh node snapshot.

Importing fresh node snapshots is the most often reason, actually. This happens because snapshots usually don't contain context data, but only blocks data, so when you import a snapshot you import just the block storage, but the context storage is empty for all blocks from the genesis to the last block in the snapshot.

Why TzKT faces this issue

Let's say you have an indexer that stopped syncing at the block 1000 and then you import a fresh node snapshot which is on the block 1010. When you start the indexer, it tries to fetch the block 1001, but your node doesn't have context for it (due to snapshot), and actually for all blocks older than 1010. So the indexer just can't receive necessary data, and this is what you have faced.

How to solve it

The solution is pretty simple:

  1. Import the latest TzKT snapshot (e.g. https://tzkt.fra1.digitaloceanspaces.com/snapshots/tzkt_v1.7_mainnet.backup).
  2. Import the node snapshot (e.g. https://snapshots-tezos.giganode.io) which is older than TzKT's one.

You can check at which block the indexer is by calling API /v1/head, but normally, TzKT exports snapshots daily, so it should be ok to just use node snapshots that are 2-3 days old. This way you will have context for recent blocks, needed to make TzKT catch up with the node.

from tzkt.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.