Git Product home page Git Product logo

cumulus's Introduction

Cumulus ☁️

Doc

This repository contains both the Cumulus SDK and also specific chains implemented on top of this SDK.

Cumulus SDK

A set of tools for writing Substrate-based Polkadot parachains. Refer to the included overview for architectural details, and the Connect to relay and parachain tutorials for a guided walk-through of using these tools.

It's easy to write blockchains using Substrate, and the overhead of writing parachains' distribution, p2p, database, and synchronization layers should be just as low. This project aims to make it easy to write parachains for Polkadot by leveraging the power of Substrate.

Cumulus clouds are shaped sort of like dots; together they form a system that is intricate, beautiful and functional.

Consensus

parachain-consensus is a consensus engine for Substrate that follows a Polkadot relay chain. This will run a Polkadot node internally, and dictate to the client and synchronization algorithms which chain to follow, finalize, and treat as best.

Collator

A Polkadot collator for the parachain is implemented by the polkadot-parachain binary (previously called polkadot-collator).

Installation

Before building Cumulus SDK based nodes / runtimes prepare your environment by following Substrate installation instructions.

Statemint 🪙

This repository also contains the Statemint runtime (as well as the canary runtime Statemine and the test runtime Westmint). Statemint is a common good parachain providing an asset store for the Polkadot ecosystem.

Build & Launch a Node

To run a Statemine or Westmint node (Statemint is not deployed, yet) you will need to compile the polkadot-parachain binary:

cargo build --release --locked -p polkadot-parachain

Once the executable is built, launch the parachain node via:

CHAIN=westmint # or statemine
./target/release/polkadot-parachain --chain $CHAIN

Refer to the setup instructions below to run a local network for development.

Contracts 📝

See the contracts-rococo readme for details.

Rococo 👑

Rococo is becoming a Community Parachain Testbed for parachain teams in the Polkadot ecosystem. It supports multiple parachains with the differentiation of long-term connections and recurring short-term connections, to see which parachains are currently connected and how long they will be connected for see here.

Rococo is an elaborate style of design and the name describes the painstaking effort that has gone into this project.

Build & Launch Rococo Collators

Collators are similar to validators in the relay chain. These nodes build the blocks that will eventually be included by the relay chain for a parachain.

To run a Rococo collator you will need to compile the following binary:

cargo build --release --locked -p polkadot-parachain

Otherwise you can compile it with Parity CI docker image:

docker run --rm -it -w /shellhere/cumulus \
                    -v $(pwd):/shellhere/cumulus \
                    paritytech/ci-linux:production cargo build --release --locked -p polkadot-parachain
sudo chown -R $(id -u):$(id -g) target/

If you want to reproduce other steps of CI process you can use the following guide.

Once the executable is built, launch collators for each parachain (repeat once each for chain tick, trick, track):

./target/release/polkadot-parachain --chain $CHAIN --validator

Parachains

The network uses horizontal message passing (HRMP) to enable communication between parachains and the relay chain and, in turn, between parachains. This means that every message is sent to the relay chain, and from the relay chain to its destination parachain.

Local Setup

Launch a local setup including a Relay Chain and a Parachain.

Launch the Relay Chain

# Clone
git clone https://github.com/paritytech/polkadot
cd polkadot

# Compile Polkadot with the real overseer feature
cargo build --release

# Generate a raw chain spec
./target/release/polkadot build-spec --chain rococo-local --disable-default-bootnode --raw > rococo-local-cfde.json

# Alice
./target/release/polkadot --chain rococo-local-cfde.json --alice --tmp

# Bob (In a separate terminal)
./target/release/polkadot --chain rococo-local-cfde.json --bob --tmp --port 30334

Launch the Parachain

# Clone
git clone https://github.com/paritytech/cumulus
cd cumulus

# Compile
cargo build --release

# Export genesis state
./target/release/polkadot-parachain export-genesis-state > genesis-state

# Export genesis wasm
./target/release/polkadot-parachain export-genesis-wasm > genesis-wasm

# Collator1
./target/release/polkadot-parachain --collator --alice --force-authoring --tmp --port 40335 --ws-port 9946 -- --execution wasm --chain ../polkadot/rococo-local-cfde.json --port 30335

# Collator2
./target/release/polkadot-parachain --collator --bob --force-authoring --tmp --port 40336 --ws-port 9947 -- --execution wasm --chain ../polkadot/rococo-local-cfde.json --port 30336

# Parachain Full Node 1
./target/release/polkadot-parachain --tmp --port 40337 --ws-port 9948 -- --execution wasm --chain ../polkadot/rococo-local-cfde.json --port 30337

Register the parachain

image

Containerize

After building polkadot-parachain with cargo or with Parity CI image as documented in this chapter, the following will allow producing a new docker image where the compiled binary is injected:

./docker/scripts/build-injected-image.sh

Alternatively, you can build an image with a builder pattern:

docker build --tag $OWNER/$IMAGE_NAME --file ./docker/polkadot-parachain_builder.Containerfile .

You may then run your new container:

```bash
docker run --rm -it $OWNER/$IMAGE_NAME --collator --tmp --execution wasm --chain /specs/westmint.json

cumulus's People

Watchers

 avatar  avatar

cumulus's Issues

Polkadot statemine-v99 Release checklist

Cumulus Release Checklist

This is the release checklist for Polkadot statemine-v99. All following
checks should be completed before publishing a new release. The current release candidate can be
checked out with git checkout release-statemine-v99

Runtime Releases

These checks should be performed on the codebase prior to forking to a release-
candidate branch.

  • Verify spec_version has been incremented since the
    last release for any native runtimes from any existing use on public
    (non-private/test) networks.
  • Verify previously completed migrations are
    removed for any public (non-private/test) networks.
  • Verify pallet and extrinsic ordering has stayed
    the same. Bump transaction_version if not.
  • Verify new extrinsics have been correctly whitelisted/blacklisted for
    proxy filters.
  • Verify benchmarks have been updated for any modified
    runtime logic.

The following checks can be performed after we have forked off to the release-
candidate branch or started an additional release candidate branch (rc-2, rc-3, etc)

  • Verify new migrations complete successfully, and the
    runtime state is correctly updated for any public (non-private/test)
    networks.
  • Verify Polkadot JS API are up to date with the latest
    runtime changes.
  • Push runtime upgrade to Westend and verify network stability.

All Releases

Notes

Burn In

Ensure that Parity DevOps has run the new release on Westend, Kusama, and
Polkadot validators for at least 12 hours prior to publishing the release.

Build Artifacts

Add any necessary assets to the release. They should include:

  • Linux binary
  • GPG signature of the Linux binary
  • SHA256 of binary
  • Source code
  • Wasm binaries of any runtimes

Release notes

The release notes should list:

  • The priority of the release (i.e., how quickly users should upgrade) - this is
    based on the max priority of any client changes.
  • Which native runtimes and their versions are included
  • The proposal hashes of the runtimes as built with
    srtool
  • Any changes in this release that are still awaiting audit

The release notes may also list:

  • Free text at the beginning of the notes mentioning anything important
    regarding this release
  • Notable changes (those labelled with B[1-9]-* labels) separated into sections

Spec Version

A runtime upgrade must bump the spec number. This may follow a pattern with the
client release (e.g. runtime v12 corresponds to v0.8.12, even if the current
runtime is not v11).

Old Migrations Removed

Any previous on_runtime_upgrade functions from old upgrades must be removed
to prevent them from executing a second time. The on_runtime_upgrade function
can be found in runtime/<runtime>/src/lib.rs.

New Migrations

Ensure that any migrations that are required due to storage or logic changes
are included in the on_runtime_upgrade function of the appropriate pallets.

Extrinsic Ordering

Offline signing libraries depend on a consistent ordering of call indices and
functions. Compare the metadata of the current and new runtimes and ensure that
the module index, call index tuples map to the same set of functions. In case
of a breaking change, increase transaction_version.

To verify the order has not changed, you may manually start the following Github Action. It takes around a minute to run and will produce the report as artifact you need to manually check.

The things to look for in the output are lines like:

  • [Identity] idx 28 -> 25 (calls 15) - indicates the index for Identity has changed
  • [+] Society, Recovery - indicates the new version includes 2 additional modules/pallets.
  • If no indices have changed, every modules line should look something like [Identity] idx 25 (calls 15)

Note: Adding new functions to the runtime does not constitute a breaking change
as long as the indexes did not change.

Proxy Filtering

The runtime contains proxy filters that map proxy types to allowable calls. If
the new runtime contains any new calls, verify that the proxy filters are up to
date to include them.

Benchmarks

There are three benchmarking machines reserved for updating the weights at
release-time. To initialise a benchmark run for each production runtime
(westend, kusama, polkadot):

  • Go to https://gitlab.parity.io/parity/polkadot/-/pipelines?page=1&scope=branches&ref=master
  • Click the link to the last pipeline run for master
  • Start each of the manual jobs:
    • 'update_westend_weights'
    • 'update_polkadot_weights'
    • 'update_kusama_weights'
  • When these jobs have completed (it takes a few hours), a git PATCH file will
    be available to download as an artifact.
  • On your local machine, branch off master
  • Download the patch file and apply it to your branch with git patch patchfile.patch
  • Commit the changes to your branch and submit a PR against master
  • The weights should be (Currently manually) checked to make sure there are no
    big outliers (i.e., twice or half the weight).

Polkadot JS

Ensure that a release of Polkadot JS API contains any new types or
interfaces necessary to interact with the new runtime.

Cumulus foobar-v99.0.1 Release checklist

Release Checklist

This is the release checklist for Statemine v6 and the initial version of
Statemint 🎉 (also v6 because of code parity with Statemine).
All following checks should be completed before publishing a
new release of the Statemine runtime. The current release candidate can be
checked out with git checkout release-statemine-v6.

Runtime Releases

These checks should be performed on the codebase.

  • Verify spec_version has been incremented since the
    last release for any native runtimes from any existing use on public
    (non-private/test) networks.
  • Verify previously completed migrations are
    removed for any public (non-private/test) networks. @apopiak
    • No migrations added in the last release that would need to be removed.
  • Verify pallet and extrinsic ordering as well as SignedExtensions have stayed
    the same. Bump transaction_version if not. @NachoPal
  • Verify new extrinsics have been correctly whitelisted/blacklisted for
    proxy filters. @apopiak
    • No new extrinsics.
  • Verify benchmarks have been updated for any modified
    runtime logic. @NachoPal
    • Verify the weights are up-to-date. @NachoPal
  • Verify that the various pieces of XCM config are sane. @apopiak

The following checks can be performed after we have forked off to the release-
candidate branch or started an additional release candidate branch (rc-2, rc-3, etc)

  • Verify new migrations complete successfully, and the
    runtime state is correctly updated for any public (non-private/test)
    networks.
  • Verify Polkadot JS API are up to date with the latest
    runtime changes.
  • Push runtime upgrade to Westmint and verify network stability.

Initial Statemint Release

On top of the regular runtime release process, this release will also need to cover the following points:

  • genesis configuration for Statemint @apopiak
  • determine setStorage call for governance to set the initial state Statemint should have after the upgrade (replacing the usual genesis process, but making use of the genesis config) @apopiak
  • create adusted chain spec that will be used for syncing Statemint @NachoPal
  • test the upgrade process (including setStorage calls) @apopiak

All Releases

Notes

Burn In

Ensure that Parity DevOps has run the new release on Westmint and Statemine collators for 12h prior to publishing the release.

Build Artifacts

Add any necessary assets to the release. They should include:

  • Linux binary
  • GPG signature of the Linux binary
  • SHA256 of binary
  • Source code
  • Wasm binaries of any runtimes

Release notes

The release notes should list:

  • The priority of the release (i.e., how quickly users should upgrade) - this is
    based on the max priority of any client changes.
  • Which native runtimes and their versions are included
  • The proposal hashes of the runtimes as built with
    srtool
  • Any changes in this release that are still awaiting audit

The release notes may also list:

  • Free text at the beginning of the notes mentioning anything important
    regarding this release
  • Notable changes separated into sections.

Spec Version

A runtime upgrade must bump the spec number. This may follow a pattern with the
client release (e.g. runtime v12 corresponds to v0.8.12, even if the current
runtime is not v11).

Old Migrations Removed

Previous on_runtime_upgrade functions from old upgrades should be removed.

New Migrations

Ensure that any migrations that are required due to storage or logic changes
are included in the on_runtime_upgrade function of the appropriate pallets.

Extrinsic Ordering & Storage

Offline signing libraries depend on a consistent ordering of call indices and
functions. Compare the metadata of the current and new runtimes and ensure that
the module index, call index tuples map to the same set of functions. It also checks if there have been any changes in storage. In case of a breaking change, increase transaction_version.

To verify the order has not changed, manually start the following Github Action. It takes around a minute to run and will produce the report as artifact you need to manually check.

To run it, in the Run Workflow dropdown:

  1. Use workflow from: to ignore, leave master as default
  2. The WebSocket url of the reference node:
    • Statemine: wss://kusama-statemine-rpc.paritytech.net
    • Westmint: wss://westmint-rpc.polkadot.io
  3. A url to a Linux binary for the node containing the runtime to test: Paste the URL of the latest release-candidate binary from the draft-release on Github. The binary has to previously be uploaded to S3 (Github url link to the binary is constantly changing)
  4. The name of the chain under test. Usually, you would pass a local chain:
    • Statemine: statemine-local
    • Westmint: westmint-local
  5. Click Run workflow

When the workflow is done, click on it and download the zip artifact, inside you'll find an output.txt file. The things to look for in the output are lines like:

  • [Identity] idx 28 -> 25 (calls 15) - indicates the index for Identity has changed
  • [+] Society, Recovery - indicates the new version includes 2 additional modules/pallets.
  • If no indices have changed, every modules line should look something like [Identity] idx 25 (calls 15)

Note: Adding new functions to the runtime does not constitute a breaking change
as long as the indexes did not change.

Note: Extrinsic function signatures changes (adding/removing & ordering arguments) are not caught by the job, so those changes should be reviewed "manually"

Proxy Filtering

The runtime contains proxy filters that map proxy types to allowable calls. If
the new runtime contains any new calls, verify that the proxy filters are up to
date to include them.

Benchmarks

Until paritytech#631 is done, running the benchmarks is a manual process:

  1. Connect to the bechmarking machine 149.202.69.202
  2. Make sure no one else is using the machine with htop check
  3. Pull in the branch of Cumulus that has the version of Statemine you want to release
  4. Recompile cargo build --release --features runtime-benchmarks
  5. From the root directory run nohup ./scripts/benchmarks.sh & (it will take quite a few hours)
  6. Checkout in your local machine to the branch of cumulus that has the version of Statemine you want to release
  7. scp from the host to your local machine the weights for Statemine, Westmint and Statemint you'll find in:
    • /polkadot-parachains/statemine/src/weights
    • /polkadot-parachains/westmint/src/weights
    • /polkadot-parachains/statemint/src/weights
  8. Commit the changes in your local and create a PR

Polkadot JS

Ensure that a release of Polkadot JS API can interact with the new runtime (mostly related to RPCs with the new metadata).

Polkadot statemine-v99 Release checklist

Cumulus Release Checklist

This is the release checklist for Polkadot statemine-v99. All following
checks should be completed before publishing a new release. The current release candidate can be
checked out with git checkout release-statemine-v99

Runtime Releases

These checks should be performed on the codebase prior to forking to a release-
candidate branch.

  • Verify spec_version has been incremented since the
    last release for any native runtimes from any existing use on public
    (non-private/test) networks.
  • Verify previously completed migrations are
    removed for any public (non-private/test) networks.
  • Verify pallet and extrinsic ordering has stayed
    the same. Bump transaction_version if not.
  • Verify new extrinsics have been correctly whitelisted/blacklisted for
    proxy filters.
  • Verify benchmarks have been updated for any modified
    runtime logic.

The following checks can be performed after we have forked off to the release-
candidate branch or started an additional release candidate branch (rc-2, rc-3, etc)

  • Verify new migrations complete successfully, and the
    runtime state is correctly updated for any public (non-private/test)
    networks.
  • Verify Polkadot JS API are up to date with the latest
    runtime changes.
  • Push runtime upgrade to Westend and verify network stability.

All Releases

Notes

Burn In

Ensure that Parity DevOps has run the new release on Westend, Kusama, and
Polkadot validators for at least 12 hours prior to publishing the release.

Build Artifacts

Add any necessary assets to the release. They should include:

  • Linux binary
  • GPG signature of the Linux binary
  • SHA256 of binary
  • Source code
  • Wasm binaries of any runtimes

Release notes

The release notes should list:

  • The priority of the release (i.e., how quickly users should upgrade) - this is
    based on the max priority of any client changes.
  • Which native runtimes and their versions are included
  • The proposal hashes of the runtimes as built with
    srtool
  • Any changes in this release that are still awaiting audit

The release notes may also list:

  • Free text at the beginning of the notes mentioning anything important
    regarding this release
  • Notable changes (those labelled with B[1-9]-* labels) separated into sections

Spec Version

A runtime upgrade must bump the spec number. This may follow a pattern with the
client release (e.g. runtime v12 corresponds to v0.8.12, even if the current
runtime is not v11).

Old Migrations Removed

Any previous on_runtime_upgrade functions from old upgrades must be removed
to prevent them from executing a second time. The on_runtime_upgrade function
can be found in runtime/<runtime>/src/lib.rs.

New Migrations

Ensure that any migrations that are required due to storage or logic changes
are included in the on_runtime_upgrade function of the appropriate pallets.

Extrinsic Ordering

Offline signing libraries depend on a consistent ordering of call indices and
functions. Compare the metadata of the current and new runtimes and ensure that
the module index, call index tuples map to the same set of functions. In case
of a breaking change, increase transaction_version.

To verify the order has not changed, you may manually start the following Github Action. It takes around a minute to run and will produce the report as artifact you need to manually check.

The things to look for in the output are lines like:

  • [Identity] idx 28 -> 25 (calls 15) - indicates the index for Identity has changed
  • [+] Society, Recovery - indicates the new version includes 2 additional modules/pallets.
  • If no indices have changed, every modules line should look something like [Identity] idx 25 (calls 15)

Note: Adding new functions to the runtime does not constitute a breaking change
as long as the indexes did not change.

Proxy Filtering

The runtime contains proxy filters that map proxy types to allowable calls. If
the new runtime contains any new calls, verify that the proxy filters are up to
date to include them.

Benchmarks

There are three benchmarking machines reserved for updating the weights at
release-time. To initialise a benchmark run for each production runtime
(westend, kusama, polkadot):

  • Go to https://gitlab.parity.io/parity/polkadot/-/pipelines?page=1&scope=branches&ref=master
  • Click the link to the last pipeline run for master
  • Start each of the manual jobs:
    • 'update_westend_weights'
    • 'update_polkadot_weights'
    • 'update_kusama_weights'
  • When these jobs have completed (it takes a few hours), a git PATCH file will
    be available to download as an artifact.
  • On your local machine, branch off master
  • Download the patch file and apply it to your branch with git patch patchfile.patch
  • Commit the changes to your branch and submit a PR against master
  • The weights should be (Currently manually) checked to make sure there are no
    big outliers (i.e., twice or half the weight).

Polkadot JS

Ensure that a release of Polkadot JS API contains any new types or
interfaces necessary to interact with the new runtime.

Polkadot statemine-v99 Release checklist

Cumulus Release Checklist

This is the release checklist for Polkadot statemine-v99. All following
checks should be completed before publishing a new release. The current release candidate can be
checked out with git checkout release-statemine-v99

Runtime Releases

These checks should be performed on the codebase prior to forking to a release-
candidate branch.

  • Verify spec_version has been incremented since the
    last release for any native runtimes from any existing use on public
    (non-private/test) networks.
  • Verify previously completed migrations are
    removed for any public (non-private/test) networks.
  • Verify pallet and extrinsic ordering has stayed
    the same. Bump transaction_version if not.
  • Verify new extrinsics have been correctly whitelisted/blacklisted for
    proxy filters.
  • Verify benchmarks have been updated for any modified
    runtime logic.

The following checks can be performed after we have forked off to the release-
candidate branch or started an additional release candidate branch (rc-2, rc-3, etc)

  • Verify new migrations complete successfully, and the
    runtime state is correctly updated for any public (non-private/test)
    networks.
  • Verify Polkadot JS API are up to date with the latest
    runtime changes.
  • Push runtime upgrade to Westend and verify network stability.

All Releases

Notes

Burn In

Ensure that Parity DevOps has run the new release on Westend, Kusama, and
Polkadot validators for at least 12 hours prior to publishing the release.

Build Artifacts

Add any necessary assets to the release. They should include:

  • Linux binary
  • GPG signature of the Linux binary
  • SHA256 of binary
  • Source code
  • Wasm binaries of any runtimes

Release notes

The release notes should list:

  • The priority of the release (i.e., how quickly users should upgrade) - this is
    based on the max priority of any client changes.
  • Which native runtimes and their versions are included
  • The proposal hashes of the runtimes as built with
    srtool
  • Any changes in this release that are still awaiting audit

The release notes may also list:

  • Free text at the beginning of the notes mentioning anything important
    regarding this release
  • Notable changes (those labelled with B[1-9]-* labels) separated into sections

Spec Version

A runtime upgrade must bump the spec number. This may follow a pattern with the
client release (e.g. runtime v12 corresponds to v0.8.12, even if the current
runtime is not v11).

Old Migrations Removed

Any previous on_runtime_upgrade functions from old upgrades must be removed
to prevent them from executing a second time. The on_runtime_upgrade function
can be found in runtime/<runtime>/src/lib.rs.

New Migrations

Ensure that any migrations that are required due to storage or logic changes
are included in the on_runtime_upgrade function of the appropriate pallets.

Extrinsic Ordering

Offline signing libraries depend on a consistent ordering of call indices and
functions. Compare the metadata of the current and new runtimes and ensure that
the module index, call index tuples map to the same set of functions. In case
of a breaking change, increase transaction_version.

To verify the order has not changed, you may manually start the following Github Action. It takes around a minute to run and will produce the report as artifact you need to manually check.

The things to look for in the output are lines like:

  • [Identity] idx 28 -> 25 (calls 15) - indicates the index for Identity has changed
  • [+] Society, Recovery - indicates the new version includes 2 additional modules/pallets.
  • If no indices have changed, every modules line should look something like [Identity] idx 25 (calls 15)

Note: Adding new functions to the runtime does not constitute a breaking change
as long as the indexes did not change.

Proxy Filtering

The runtime contains proxy filters that map proxy types to allowable calls. If
the new runtime contains any new calls, verify that the proxy filters are up to
date to include them.

Benchmarks

There are three benchmarking machines reserved for updating the weights at
release-time. To initialise a benchmark run for each production runtime
(westend, kusama, polkadot):

  • Go to https://gitlab.parity.io/parity/polkadot/-/pipelines?page=1&scope=branches&ref=master
  • Click the link to the last pipeline run for master
  • Start each of the manual jobs:
    • 'update_westend_weights'
    • 'update_polkadot_weights'
    • 'update_kusama_weights'
  • When these jobs have completed (it takes a few hours), a git PATCH file will
    be available to download as an artifact.
  • On your local machine, branch off master
  • Download the patch file and apply it to your branch with git patch patchfile.patch
  • Commit the changes to your branch and submit a PR against master
  • The weights should be (Currently manually) checked to make sure there are no
    big outliers (i.e., twice or half the weight).

Polkadot JS

Ensure that a release of Polkadot JS API contains any new types or
interfaces necessary to interact with the new runtime.

Cumulus statemint-v6.0.1 Release checklist

Release Checklist

Runtime Releases

These checks should be performed on the codebase.

  • Verify spec_version has been incremented since the
    last release for any native runtimes from any existing use on public
    (non-private/test) networks.
  • Verify previously completed migrations are
    removed for any public (non-private/test) networks.
    • No migrations added in the last release that would need to be removed.
  • Verify pallet and extrinsic ordering as well as SignedExtensions have stayed
    the same. Bump transaction_version if not.
  • Verify new extrinsics have been correctly whitelisted/blacklisted for
    proxy filters.
    • No new extrinsics.
  • Verify benchmarks have been updated for any modified
    runtime logic.
    • Verify the weights are up-to-date.
  • Verify that the various pieces of XCM config are sane.

The following checks can be performed after we have forked off to the release-
candidate branch or started an additional release candidate branch (rc-2, rc-3, etc)

  • Verify new migrations complete successfully, and the
    runtime state is correctly updated for any public (non-private/test)
    networks.
  • Run integration tests.
    • Teleport Relay -> Statemin* and back.
    • Create asset (if applicable), mint and transfer
  • Push runtime upgrade to Westmint and verify network stability.

All Releases


Notes

Burn In

Ensure that Parity DevOps has run the new release on Westmint and Statemine collators for 12h prior to publishing the release.

Build Artifacts

Add any necessary assets to the release. They should include:

  • Linux binary
  • GPG signature of the Linux binary
  • SHA256 of binary
  • Source code
  • Wasm binaries of any runtimes

Release notes

The release notes should list:

  • The priority of the release (i.e., how quickly users should upgrade) - this is
    based on the max priority of any client changes.
  • Which native runtimes and their versions are included
  • The proposal hashes of the runtimes as built with
    srtool
  • Any changes in this release that are still awaiting audit

The release notes may also list:

  • Free text at the beginning of the notes mentioning anything important
    regarding this release
  • Notable changes separated into sections.

Spec Version

A runtime upgrade must bump the spec number. This may follow a pattern with the
client release (e.g. runtime v12 corresponds to v0.8.12, even if the current
runtime is not v11).

Runtime version bump between RCs

The clients need to be aware of runtime changes. However, we do not want to bump the
spec_version for every single release candidate. Instead, we can bump the impl field of the version
to signal the change to the client.

Old Migrations Removed

Previous on_runtime_upgrade functions from old upgrades should be removed.

New Migrations

Ensure that any migrations that are required due to storage or logic changes
are included in the on_runtime_upgrade function of the appropriate pallets.

Extrinsic Ordering & Storage

Offline signing libraries depend on a consistent ordering of call indices and
functions. Compare the metadata of the current and new runtimes and ensure that
the module index, call index tuples map to the same set of functions. It also checks if there have been any changes in storage. In case of a breaking change, increase transaction_version.

To verify the order has not changed, manually start the following Github Action. It takes around a minute to run and will produce the report as artifact you need to manually check.

To run it, in the Run Workflow dropdown:

  1. Use workflow from: to ignore, leave master as default
  2. The WebSocket url of the reference node:
    • Statemine: wss://kusama-statemine-rpc.paritytech.net
    • Westmint: wss://westmint-rpc.polkadot.io
  3. A url to a Linux binary for the node containing the runtime to test: Paste the URL of the latest release-candidate binary from the draft-release on Github. The binary has to previously be uploaded to S3 (Github url link to the binary is constantly changing)
  4. The name of the chain under test. Usually, you would pass a local chain:
    • Statemine: statemine-local
    • Westmint: westmint-local
  5. Click Run workflow

When the workflow is done, click on it and download the zip artifact, inside you'll find an output.txt file. The things to look for in the output are lines like:

  • [Identity] idx 28 -> 25 (calls 15) - indicates the index for Identity has changed
  • [+] Society, Recovery - indicates the new version includes 2 additional modules/pallets.
  • If no indices have changed, every modules line should look something like [Identity] idx 25 (calls 15)

Note: Adding new functions to the runtime does not constitute a breaking change
as long as the indexes did not change.

Note: Extrinsic function signatures changes (adding/removing & ordering arguments) are not caught by the job, so those changes should be reviewed "manually"

Proxy Filtering

The runtime contains proxy filters that map proxy types to allowable calls. If
the new runtime contains any new calls, verify that the proxy filters are up to
date to include them.

Benchmarks

Until paritytech#631 is done, running the benchmarks is a manual process:

  1. Connect to the bechmarking machine
  2. Make sure no one else is using the machine with htop check
  3. Pull in the branch of Cumulus that has the version of Statemine you want to release
  4. Recompile cargo build --release --features runtime-benchmarks
  5. From the root directory run nohup ./scripts/benchmarks.sh & (it will take quite a few hours)
  6. Checkout in your local machine to the branch of cumulus that has the version of Statemine you want to release
  7. scp from the host to your local machine the weights for Statemine, Westmint and Statemint you'll find in:
    • /polkadot-parachains/statemine/src/weights
    • /polkadot-parachains/westmint/src/weights
    • /polkadot-parachains/statemint/src/weights
  8. Commit the changes in your local and create a PR

Cumulus parachains-v8.0.0 Release checklist

Release Checklist

Runtime Releases

These checks should be performed on the codebase.

  • Verify spec_version has been incremented since the
    last release for any native runtimes from any existing use on public
    (non-private/test) networks.
  • Verify previously completed migrations are
    removed for any public (non-private/test) networks.
    • No migrations added in the last release that would need to be removed.
  • Verify pallet and extrinsic ordering as well as SignedExtensions have stayed
    the same. Bump transaction_version if not.
  • Verify new extrinsics have been correctly whitelisted/blacklisted for
    proxy filters.
    • No new extrinsics.
  • Verify benchmarks have been updated for any modified
    runtime logic.
    • Verify the weights are up-to-date.
  • Verify that the various pieces of XCM config are sane.

The following checks can be performed after we have forked off to the release-
candidate branch or started an additional release candidate branch (rc-2, rc-3, etc)

  • Verify new migrations complete successfully, and the
    runtime state is correctly updated for any public (non-private/test)
    networks.
  • Run integration tests.
    • Teleport Relay -> Statemin* and back.
    • Create asset (if applicable), mint and transfer
  • Push runtime upgrade to Westmint and verify network stability.

All Releases


Notes

Burn In

Ensure that Parity DevOps has run the new release on Westmint and Statemine collators for 12h prior to publishing the release.

Build Artifacts

Add any necessary assets to the release. They should include:

  • Linux binary
  • GPG signature of the Linux binary
  • SHA256 of binary
  • Source code
  • Wasm binaries of any runtimes

Release notes

The release notes should list:

  • The priority of the release (i.e., how quickly users should upgrade) - this is
    based on the max priority of any client changes.
  • Which native runtimes and their versions are included
  • The proposal hashes of the runtimes as built with
    srtool
  • Any changes in this release that are still awaiting audit

The release notes may also list:

  • Free text at the beginning of the notes mentioning anything important
    regarding this release
  • Notable changes separated into sections.

Spec Version

A runtime upgrade must bump the spec number. This may follow a pattern with the
client release (e.g. runtime v12 corresponds to v0.8.12, even if the current
runtime is not v11).

Runtime version bump between RCs

The clients need to be aware of runtime changes. However, we do not want to bump the
spec_version for every single release candidate. Instead, we can bump the impl field of the version
to signal the change to the client.

Old Migrations Removed

Previous on_runtime_upgrade functions from old upgrades should be removed.

New Migrations

Ensure that any migrations that are required due to storage or logic changes
are included in the on_runtime_upgrade function of the appropriate pallets.

Extrinsic Ordering & Storage

Offline signing libraries depend on a consistent ordering of call indices and
functions. Compare the metadata of the current and new runtimes and ensure that
the module index, call index tuples map to the same set of functions. It also checks if there have been any changes in storage. In case of a breaking change, increase transaction_version.

To verify the order has not changed, manually start the following Github Action. It takes around a minute to run and will produce the report as artifact you need to manually check.

To run it, in the Run Workflow dropdown:

  1. Use workflow from: to ignore, leave master as default
  2. The WebSocket url of the reference node:
    • Statemine: wss://kusama-statemine-rpc.paritytech.net
    • Westmint: wss://westmint-rpc.polkadot.io
  3. A url to a Linux binary for the node containing the runtime to test: Paste the URL of the latest release-candidate binary from the draft-release on Github. The binary has to previously be uploaded to S3 (Github url link to the binary is constantly changing)
  4. The name of the chain under test. Usually, you would pass a local chain:
    • Statemine: statemine-local
    • Westmint: westmint-local
  5. Click Run workflow

When the workflow is done, click on it and download the zip artifact, inside you'll find an output.txt file. The things to look for in the output are lines like:

  • [Identity] idx 28 -> 25 (calls 15) - indicates the index for Identity has changed
  • [+] Society, Recovery - indicates the new version includes 2 additional modules/pallets.
  • If no indices have changed, every modules line should look something like [Identity] idx 25 (calls 15)

Note: Adding new functions to the runtime does not constitute a breaking change
as long as the indexes did not change.

Note: Extrinsic function signatures changes (adding/removing & ordering arguments) are not caught by the job, so those changes should be reviewed "manually"

Proxy Filtering

The runtime contains proxy filters that map proxy types to allowable calls. If
the new runtime contains any new calls, verify that the proxy filters are up to
date to include them.

Benchmarks

Until paritytech#631 is done, running the benchmarks is a manual process:

  1. Connect to the bechmarking machine
  2. Make sure no one else is using the machine with htop check
  3. Pull in the branch of Cumulus that has the version of Statemine you want to release
  4. Recompile cargo build --profile production --locked --features runtime-benchmarks
  5. From the root directory run nohup ./scripts/benchmarks.sh & (it will take quite a few hours)
  6. Checkout in your local machine to the branch of cumulus that has the version of Statemine you want to release
  7. scp from the host to your local machine the weights for Statemine, Westmint and Statemint you'll find in:
    • /polkadot-parachains/statemine/src/weights
    • /polkadot-parachains/westmint/src/weights
    • /polkadot-parachains/statemint/src/weights
  8. Commit the changes in your local and create a PR

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.