Git Product home page Git Product logo

ethereum-package's Introduction

Important recent update notes - temporary note

The ethereum-package has been moved to the ethpandaops organization.

The new repository is located at github.com/ethpandaops/ethereum-package. For all your references please replace kurtosis-tech with ethpandaops.

If you would like to use the latest release of the package, released by kurtosis-tech, please refer to using the tag v3.1.0.

Ethereum Package

Run of the Ethereum Network Package

This is a Kurtosis package that will spin up a private Ethereum testnet over Docker or Kubernetes with multi-client support, Flashbot's mev-boost infrastructure for PBS-related testing/validation, and other useful network tools (transaction spammer, monitoring tools, etc). Kurtosis packages are entirely reproducible and composable, so this will work the same way over Docker or Kubernetes, in the cloud or locally on your machine.

You now have the ability to spin up a private Ethereum testnet or public devnet/testnet (e.g. Goerli, Holesky, Sepolia, dencun-devnet-12, verkle-gen-devnet-2 etc) with a single command. This package is designed to be used for testing, validation, and development of Ethereum clients, and is not intended for production use. For more details check network_params.network in the configuration section.

Specifically, this package will:

  1. Generate Execution Layer (EL) & Consensus Layer (CL) genesis information using the Ethereum genesis generator.
  2. Configure & bootstrap a network of Ethereum nodes of n size using the genesis data generated above
  3. Spin up a transaction spammer to send fake transactions to the network
  4. Spin up and connect a testnet verifier
  5. Spin up a Grafana and Prometheus instance to observe the network
  6. Spin up a Blobscan instance to analyze blob transactions (EIP-4844)

Optional features (enabled via flags or parameter files at runtime):

  • Block until the Beacon nodes finalize an epoch (i.e. finalized_epoch > 0)
  • Spin up & configure parameters for the infrastructure behind Flashbot's implementation of PBS using mev-boost, in either full or mock mode. More details here.
  • Spin up & connect the network to a beacon metrics gazer service to collect network-wide participation metrics.
  • Spin up and connect a JSON RPC Snooper to the network log responses & requests between the EL engine API and the CL client.
  • Specify extra parameters to be passed in for any of the: CL client Beacon, and CL client validator, and/or EL client containers
  • Specify the required parameters for the nodes to reach an external block building network
  • Generate keystores for each node in parallel

Quickstart

Open in Gitpod

  1. Install Docker & start the Docker Daemon if you haven't done so already

  2. Install the Kurtosis CLI, or upgrade it to the latest version if it's already installed

  3. Run the package with default configurations from the command line:

    kurtosis run --enclave my-testnet github.com/ethpandaops/ethereum-package

Run with your own configuration

Kurtosis packages are parameterizable, meaning you can customize your network and its behavior to suit your needs by storing parameters in a file that you can pass in at runtime like so:

kurtosis run --enclave my-testnet github.com/ethpandaops/ethereum-package --args-file network_params.yaml

Where network_params.yaml contains the parameters for your network in your home directory.

Run on Kubernetes

Kurtosis packages work the same way over Docker or on Kubernetes. Please visit our Kubernetes docs to learn how to spin up a private testnet on a Kubernetes cluster.

Considerations for Running on a Public Testnet with a Cloud Provider

When running on a public testnet using a cloud provider's Kubernetes cluster, there are a few important factors to consider:

  1. State Growth: The growth of the state might be faster than anticipated. This could potentially lead to issues if the default parameters become insufficient over time. It's important to monitor state growth and adjust parameters as necessary.

  2. Persistent Storage Speed: Most cloud providers provision their Kubernetes clusters with relatively slow persistent storage by default. This can cause performance issues, particularly with Execution Layer (EL) clients.

  3. Network Syncing: The disk speed provided by cloud providers may not be sufficient to sync with networks that have high demands, such as the mainnet. This could lead to syncing issues and delays.

To mitigate these issues, you can use the el_volume_size and cl_volume_size flags to override the default settings locally. This allows you to allocate more storage to the EL and CL clients, which can help accommodate faster state growth and improve syncing performance. However, keep in mind that increasing the volume size may also increase your cloud provider costs. Always monitor your usage and adjust as necessary to balance performance and cost.

For optimal performance, we recommend using a cloud provider that allows you to provision Kubernetes clusters with fast persistent storage or self hosting your own Kubernetes cluster with fast persistent storage.

Shadowforking

In order to enable shadowfork capabilities, you can use the network_params.network flag. The expected value is the name of the network you want to shadowfork followed by -shadowfork. Please note that persistent configuration parameter has to be enabled for shadowforks to work! Current limitation on k8s is it is only working on a single node cluster. For example, to shadowfork the Holesky testnet, you can use the following command:

...
network_params:
  network: "holesky-shadowfork"
persistent: true
...
Shadowforking custom verkle networks

In order to enable shadowfork capabilities for verkle networks, you need to define electra and mention verkle in the network name after shadowfork.

...
network_params:
  electra_fork_epoch: 1
  network: "holesky-shadowfork-verkle"
persistent: true
...

Taints and tolerations

It is possible to run the package on a Kubernetes cluster with taints and tolerations. This is done by adding the tolerations to the tolerations field in the network_params.yaml file. For example:

participants:
  - el_type: reth
    cl_type: teku
global_tolerations:
  - key: "node-role.kubernetes.io/master6"
    value: "true"
    operator: "Equal"
    effect: "NoSchedule"

It is possible to define toleration globally, per participant or per container. The order of precedence is as follows:

  1. Container (el_tolerations, cl_tolerations, vc_tolerations)
  2. Participant (tolerations)
  3. Global (global_tolerations)

This feature is only available for Kubernetes. To learn more about taints and tolerations, please visit the Kubernetes documentation.

Tear down

The testnet will reside in an enclave - an isolated, ephemeral environment. The enclave and its contents (e.g. running containers, files artifacts, etc) will persist until torn down. You can remove an enclave and its contents with:

kurtosis enclave rm -f my-testnet

Management

The Kurtosis CLI can be used to inspect and interact with the network.

For example, if you need shell access, simply run:

kurtosis service shell my-testnet $SERVICE_NAME

And if you need the logs for a service, simply run:

kurtosis service logs my-testnet $SERVICE_NAME

Check out the full list of CLI commands here

Debugging

To grab the genesis files for the network, simply run:

kurtosis files download my-testnet $FILE_NAME $OUTPUT_DIRECTORY

For example, to retrieve the Execution Layer (EL) genesis data, run:

kurtosis files download my-testnet el-genesis-data ~/Downloads

Basic file sharing

Apache is included in the package to allow for basic file sharing. The Apache service is started when additional services are enabled. It will expose the network-configs directory, which might needed if you want to share the network config publicly.

additional_services:
  - apache

Configuration

To configure the package behaviour, you can modify your network_params.yaml file. The full YAML schema that can be passed in is as follows with the defaults provided:

# Specification of the participants in the network
participants:
  # EL(Execution Layer) Specific flags
    # The type of EL client that should be started
    # Valid values are geth, nethermind, erigon, besu, ethereumjs, reth, nimbus-eth1
  - el_type: geth

    # The Docker image that should be used for the EL client; leave blank to use the default for the client type
    # Defaults by client:
    # - geth: ethereum/client-go:latest
    # - erigon: thorax/erigon:devel
    # - nethermind: nethermind/nethermind:latest
    # - besu: hyperledger/besu:develop
    # - reth: ghcr.io/paradigmxyz/reth
    # - ethereumjs: ethpandaops/ethereumjs:master
    # - nimbus-eth1: ethpandaops/nimbus-eth1:master
    el_image: ""

    # The log level string that this participant's EL client should log at
    # If this is emptystring then the global `logLevel` parameter's value will be translated into a string appropriate for the client (e.g. if
    # global `logLevel` = `info` then Geth would receive `3`, Besu would receive `INFO`, etc.)
    # If this is not emptystring, then this value will override the global `logLevel` setting to allow for fine-grained control
    # over a specific participant's logging
    el_log_level: ""

    # A list of optional extra env_vars the el container should spin up with
    el_extra_env_vars: {}

    # A list of optional extra labels the el container should spin up with
    # Example; el_extra_labels: {"ethereum-package.partition": "1"}
    el_extra_labels: {}

    # A list of optional extra params that will be passed to the EL client container for modifying its behaviour
    el_extra_params: []

    # A list of tolerations that will be passed to the EL client container
    # Only works with Kubernetes
    # Example: el_tolerations:
    # - key: "key"
    #   operator: "Equal"
    #   value: "value"
    #   effect: "NoSchedule"
    #   toleration_seconds: 3600
    # Defaults to empty
    el_tolerations: []

    # Persistent storage size for the EL client container (in MB)
    # Defaults to 0, which means that the default size for the client will be used
    # Default values can be found in /src/package_io/constants.star VOLUME_SIZE
    el_volume_size: 0

    # Resource management for el containers
    # CPU is milicores
    # RAM is in MB
    # Defaults are set per client
    el_min_cpu: 0
    el_max_cpu: 0
    el_min_mem: 0
    el_max_mem: 0

  # CL(Consensus Layer) Specific flags
    # The type of CL client that should be started
    # Valid values are nimbus, lighthouse, lodestar, teku, prysm, and grandine
    cl_type: lighthouse

    # The Docker image that should be used for the CL client; leave blank to use the default for the client type
    # Defaults by client:
    # - lighthouse: sigp/lighthouse:latest
    # - teku: consensys/teku:latest
    # - nimbus: statusim/nimbus-eth2:multiarch-latest
    # - prysm: gcr.io/prysmaticlabs/prysm/beacon-chain:latest
    # - lodestar: chainsafe/lodestar:next
    # - grandine: ethpandaops/grandine:develop
    cl_image: ""

    # The log level string that this participant's CL client should log at
    # If this is emptystring then the global `logLevel` parameter's value will be translated into a string appropriate for the client (e.g. if
    # global `logLevel` = `info` then Teku would receive `INFO`, Prysm would receive `info`, etc.)
    # If this is not emptystring, then this value will override the global `logLevel` setting to allow for fine-grained control
    # over a specific participant's logging
    cl_log_level: ""

    # A list of optional extra env_vars the cl container should spin up with
    cl_extra_env_vars: {}

    # A list of optional extra labels that will be passed to the CL client Beacon container.
    # Example; cl_extra_labels: {"ethereum-package.partition": "1"}
    cl_extra_labels: {}

    # A list of optional extra params that will be passed to the CL client Beacon container for modifying its behaviour
    # If the client combines the Beacon & validator nodes (e.g. Teku, Nimbus), then this list will be passed to the combined Beacon-validator node
    cl_extra_params: []

    # A list of tolerations that will be passed to the CL client container
    # Only works with Kubernetes
    # Example: el_tolerations:
    # - key: "key"
    #   operator: "Equal"
    #   value: "value"
    #   effect: "NoSchedule"
    #   toleration_seconds: 3600
    # Defaults to empty
    cl_tolerations: []

    # Persistent storage size for the CL client container (in MB)
    # Defaults to 0, which means that the default size for the client will be used
    # Default values can be found in /src/package_io/constants.star VOLUME_SIZE
    cl_volume_size: 0

    # Resource management for cl containers
    # CPU is milicores
    # RAM is in MB
    # Defaults are set per client
    cl_min_cpu: 0
    cl_max_cpu: 0
    cl_min_mem: 0
    cl_max_mem: 0

    # Whether to use a separate validator client attached to the CL client.
    # Defaults to false for clients that can run both in one process (Teku, Nimbus)
    use_separate_vc: false

  # VC (Validator Client) Specific flags
    # The type of validator client that should be used
    # Valid values are nimbus, lighthouse, lodestar, teku, and prysm
    # ( The prysm validator only works with a prysm CL client )
    # Defaults to matching the chosen CL client (cl_type)
    vc_type: ""

    # The Docker image that should be used for the separate validator client
    # Defaults by client:
    # - lighthouse: sigp/lighthouse:latest
    # - lodestar: chainsafe/lodestar:latest
    # - nimbus: statusim/nimbus-validator-client:multiarch-latest
    # - prysm: gcr.io/prysmaticlabs/prysm/validator:latest
    # - teku: consensys/teku:latest
    vc_image: ""

    # The number of validator clients to run for this participant
    # Defaults to 1
    vc_count: 1

    # The log level string that this participant's CL client should log at
    # If this is emptystring then the global `logLevel` parameter's value will be translated into a string appropriate for the client (e.g. if
    # global `logLevel` = `info` then Teku would receive `INFO`, Prysm would receive `info`, etc.)
    # If this is not emptystring, then this value will override the global `logLevel` setting to allow for fine-grained control
    # over a specific participant's logging
    vc_log_level: ""

    # A list of optional extra env_vars the vc container should spin up with
    vc_extra_env_vars: {}

    # A list of optional extra labels that will be passed to the CL client validator container.
    # Example; vc_extra_labels: {"ethereum-package.partition": "1"}
    vc_extra_labels: {}

    # A list of optional extra params that will be passed to the CL client validator container for modifying its behaviour
    # If the client combines the Beacon & validator nodes (e.g. Teku, Nimbus), then this list will also be passed to the combined Beacon-validator node
    vc_extra_params: []

    # A list of tolerations that will be passed to the validator container
    # Only works with Kubernetes
    # Example: el_tolerations:
    # - key: "key"
    #   operator: "Equal"
    #   value: "value"
    #   effect: "NoSchedule"
    #   toleration_seconds: 3600
    # Defaults to empty
    vc_tolerations: []

    # Resource management for vc containers
    # CPU is milicores
    # RAM is in MB
    # Defaults are set per client
    vc_min_cpu: 0
    vc_max_cpu: 0
    vc_min_mem: 0
    vc_max_mem: 0

    # Count of the number of validators you want to run for a given participant
    # Default to null, which means that the number of validators will be using the
    # network parameter num_validator_keys_per_node
    validator_count: null

  # Participant specific flags
    # Node selector
    # Only works with Kubernetes
    # Example: node_selectors: { "disktype": "ssd" }
    # Defaults to empty
    node_selectors: {}

    # A list of tolerations that will be passed to the EL/CL/validator containers
    # This is to be used when you don't want to specify the tolerations for each container separately
    # Only works with Kubernetes
    # Example: tolerations:
    # - key: "key"
    #   operator: "Equal"
    #   value: "value"
    #   effect: "NoSchedule"
    #   toleration_seconds: 3600
    # Defaults to empty
    tolerations: []

    # Count of nodes to spin up for this participant
    # Default to 1
    count: 1

    # Snooper can be enabled with the `snooper_enabled` flag per client or globally
    # Defaults null and then set to global snooper default (false)
    snooper_enabled: null

    # Enables Ethereum Metrics Exporter for this participant. Can be set globally.
    # Defaults null and then set to global ethereum_metrics_exporter_enabled (false)
    ethereum_metrics_exporter_enabled: null

    # Enables Xatu Sentry for this participant. Can be set globally.
    # Defaults null and then set to global xatu_sentry_enabled (false)
    xatu_sentry_enabled: null

    # Prometheus additional configuration for a given participant prometheus target.
    # Execution, beacon and validator client targets on prometheus will include this
    # configuration.
    prometheus_config:
      # Scrape interval to be used. Default to 15 seconds
      scrape_interval: 15s
      # Additional labels to be added. Default to empty
      labels: {}

    # Blobber can be enabled with the `blobber_enabled` flag per client or globally
    # Defaults to false
    blobber_enabled: false

    # Blobber extra params can be passed in to the blobber container
    # Defaults to empty
    blobber_extra_params: []

    # A set of parameters the node needs to reach an external block building network
    # If `null` then the builder infrastructure will not be instantiated
    # Example:
    #
    # "relay_endpoints": [
    #  "https://[email protected]",
    #  "https://[email protected]",
    #  "https://[email protected]",
    #  "https://[email protected]"
    # ]
    builder_network_params: null

    # Participant flag for keymanager api
    # This will open up http ports to your validator services!
    # Defaults null and then set to default global keymanager_enabled (false)
    keymanager_enabled: null

# Participants matrix creates a participant for each combination of EL, CL and VC clients
# Each EL/CL/VC item can provide the same parameters as a standard participant
participants_matrix: {}
  # el:
  #   - el_type: geth
  #   - el_type: besu
  # cl:
  #   - cl_type: prysm
  #   - cl_type: lighthouse
  # vc:
  #   - vc_type: prysm
  #   - vc_type: lighthouse


# Default configuration parameters for the network
network_params:
  # Network name, used to enable syncing of alternative networks
  # Defaults to "kurtosis"
  # You can sync any public network by setting this to the network name (e.g. "mainnet", "sepolia", "holesky")
  # You can sync any devnet by setting this to the network name (e.g. "dencun-devnet-12", "verkle-gen-devnet-2")
  network: "kurtosis"

  # The network ID of the network.
  network_id: "3151908"

  # The address of the staking contract address on the Eth1 chain
  deposit_contract_address: "0x4242424242424242424242424242424242424242"

  # Number of seconds per slot on the Beacon chain
  seconds_per_slot: 12

  # The number of validator keys that each CL validator node should get
  num_validator_keys_per_node: 64

  # This mnemonic will a) be used to create keystores for all the types of validators that we have and b) be used to generate a CL genesis.ssz that has the children
  # validator keys already preregistered as validators
  preregistered_validator_keys_mnemonic: "giant issue aisle success illegal bike spike question tent bar rely arctic volcano long crawl hungry vocal artwork sniff fantasy very lucky have athlete"

  # The number of pre-registered validators for genesis. If 0 or not specified then the value will be calculated from the participants
  preregistered_validator_count: 0

  # How long you want the network to wait before starting up
  genesis_delay: 20

  # Max churn rate for the network introduced by
  # EIP-7514 https://eips.ethereum.org/EIPS/eip-7514
  # Defaults to 8
  max_per_epoch_activation_churn_limit: 8

  # Churn limit quotient for the network
  # Defaults to 65536
  churn_limit_quotient: 65536

  # Ejection balance
  # Defaults to 16ETH
  # 16000000000 gwei
  ejection_balance: 16000000000

  # ETH1 follow distance
  # Defaults to 2048
  eth1_follow_distance: 2048

  # The number of epochs to wait validators to be able to withdraw
  # Defaults to 256 epochs ~27 hours
  min_validator_withdrawability_delay: 256

  # The period of the shard committee
  # Defaults to 256 epoch ~27 hours
  shard_committee_period: 256

  # The epoch at which the deneb/electra/eip7594(peerdas) forks are set to occur. Note: PeerDAS and Electra clients are currently
  # working on forks. So set either one of the below forks.
  deneb_fork_epoch: 0
  electra_fork_epoch: 100000000
  eip7594_fork_epoch: 100000001

  # The fork version to set if the eip7594 fork is active
  eip7594_fork_version: "0x70000038"

  # EOF activation fork epoch (EL only fork)
  # Defaults to null
  eof_activation_fork_epoch: null

  # Network sync base url for syncing public networks from a custom snapshot (mostly useful for shadowforks)
  # Defaults to "https://ethpandaops-ethereum-node-snapshots.ams3.cdn.digitaloceanspaces.com/
  # If you have a local snapshot, you can set this to the local url:
  # network_snapshot_url_base = "http://10.10.101.21:10000/snapshots/"
  # The snapshots are taken with https://github.com/ethpandaops/snapshotter
  network_sync_base_url: https://ethpandaops-ethereum-node-snapshots.ams3.cdn.digitaloceanspaces.com/

  # The number of data column sidecar subnets used in the gossipsub protocol
  data_column_sidecar_subnet_count: 32
  # Number of DataColumn random samples a node queries per slot
  samples_per_slot: 8
  # Minimum number of subnets an honest node custodies and serves samples from
  custody_requirement: 1
  # Suggested minimum peer count
  target_number_of_peers: 70

  # Preset for the network
  # Default: "mainnet"
  # Options: "mainnet", "minimal"
  # "minimal" preset will spin up a network with minimal preset. This is useful for rapid testing and development.
  # 192 seconds to get to finalized epoch vs 1536 seconds with mainnet defaults
  # Please note that minimal preset requires alternative client images.
  # For an example of minimal preset, please refer to [minimal.yaml](.github/tests/minimal.yaml)
  preset: "mainnet"

  # Preloaded contracts for the chain
  additional_preloaded_contracts: {}
  # example: To set a contract code at a certain address:
  #  "0x123463a4B065722E99115D6c222f267d9cABb524":
  #    balance: "1ETH"
  #    code: "0x1234"
  #    storage: {}
  #    nonce: 0
  #    secretKey: "0x"

  # Repository override for devnet networks
  # Default: ethpandaops
  devnet_repo: ethpandaops

# Global parameters for the network

# By default includes
# - A transaction spammer & blob spammer is launched to fake transactions sent to the network
# - Forkmon for EL will be launched
# - A prometheus will be started, coupled with grafana
# - A beacon metrics gazer will be launched
# - A light beacon chain explorer will be launched
# - Default: ["tx_spammer", "blob_spammer", "el_forkmon", "beacon_metrics_gazer", "dora"," "prometheus_grafana"]
additional_services:
  - assertoor
  - broadcaster
  - tx_spammer
  - blob_spammer
  - custom_flood
  - goomy_blob
  - el_forkmon
  - blockscout
  - beacon_metrics_gazer
  - dora
  - full_beaconchain_explorer
  - prometheus_grafana
  - blobscan
  - dugtrio
  - blutgang
  - forky
  - apache
  - tracoor

# Configuration place for dora the explorer - https://github.com/ethpandaops/dora
dora_params:
  # Dora docker image to use
  # Leave blank to use the default image according to your network params
  image: ""

  # A list of optional extra env_vars the dora container should spin up with
  env: {}

# Configuration place for transaction spammer - https://github.com/MariusVanDerWijden/tx-fuzz
tx_spammer_params:
  # A list of optional extra params that will be passed to the TX Spammer container for modifying its behaviour
  tx_spammer_extra_args: []

# Configuration place for goomy the blob spammer - https://github.com/ethpandaops/goomy-blob
goomy_blob_params:
  # A list of optional params that will be passed to the blob-spammer comamnd for modifying its behaviour
  goomy_blob_args: []

# Configuration place for the assertoor testing tool - https://github.com/ethpandaops/assertoor
assertoor_params:
  # Assertoor docker image to use
  # Leave blank to use the default image according to your network params
  image: ""

  # Check chain stability
  # This check monitors the chain and succeeds if:
  # - all clients are synced
  # - chain is finalizing for min. 2 epochs
  # - >= 98% correct target votes
  # - >= 80% correct head votes
  # - no reorgs with distance > 2 blocks
  # - no more than 2 reorgs per epoch
  run_stability_check: true

  # Check block propöosals
  # This check monitors the chain and succeeds if:
  # - all client pairs have proposed a block
  run_block_proposal_check: true

  # Run normal transaction test
  # This test generates random EOA transactions and checks inclusion with/from all client pairs
  # This test checks for:
  # - block proposals with transactions from all client pairs
  # - transaction inclusion when submitting via each client pair
  # test is done twice, first with legacy (type 0) transactions, then with dynfee (type 2) transactions
  run_transaction_test: false

  # Run blob transaction test
  # This test generates blob transactions and checks inclusion with/from all client pairs
  # This test checks for:
  # - block proposals with blobs from all client pairs
  # - blob inclusion when submitting via each client pair
  run_blob_transaction_test: false

  # Run all-opcodes transaction test
  # This test generates a transaction that triggers all EVM OPCODES once
  # This test checks for:
  # - all-opcodes transaction success
  run_opcodes_transaction_test: false

  # Run validator lifecycle test (~48h to complete)
  # This test requires exactly 500 active validator keys.
  # The test will cause a temporary chain unfinality when running.
  # This test checks:
  # - Deposit inclusion with/from all client pairs
  # - BLS Change inclusion with/from all client pairs
  # - Voluntary Exit inclusion with/from all client pairs
  # - Attester Slashing inclusion with/from all client pairs
  # - Proposer Slashing inclusion with/from all client pairs
  # all checks are done during finality & unfinality
  run_lifecycle_test: false

  # Run additional tests from external test definitions
  # Entries may be simple strings (link to the test file) or dictionaries with more flexibility
  # eg:
  #   - https://raw.githubusercontent.com/ethpandaops/assertoor/master/example/tests/block-proposal-check.yaml
  #   - file: "https://raw.githubusercontent.com/ethpandaops/assertoor/master/example/tests/block-proposal-check.yaml"
  #     config:
  #       someCustomTestConfig: "some value"
  tests: []


# If set, the package will block until a finalized epoch has occurred.
wait_for_finalization: false

# The global log level that all clients should log at
# Valid values are "error", "warn", "info", "debug", and "trace"
# This value will be overridden by participant-specific values
global_log_level: "info"

# EngineAPI Snooper global flags for all participants
# Default to false
snooper_enabled: false

# Enables Ethereum Metrics Exporter for all participants
# Defaults to false
ethereum_metrics_exporter_enabled: false

# Parallelizes keystore generation so that each node has keystores being generated in their own container
# This will result in a large number of containers being spun up than normal. We advise users to only enable this on a sufficiently large machine or in the cloud as it can be resource consuming on a single machine.
parallel_keystore_generation: false

# Disable peer scoring to prevent nodes impacted by faults from being permanently ejected from the network
# Default to false
disable_peer_scoring: false

# A list of locators for grafana dashboards to be loaded be the grafana service
grafana_additional_dashboards: []

# Whether the environment should be persistent; this is WIP and is slowly being rolled out accross services
# Note this requires Kurtosis greater than 0.85.49 to work
# Note Erigon, Besu, Teku persistence is not currently supported with docker.
# Defaults to false
persistent: false

# Supports three valeus
# Default: "null" - no mev boost, mev builder, mev flood or relays are spun up
# "mock" - mock-builder & mev-boost are spun up
# "flashbots" - mev-boost, relays, flooder and builder are all spun up, powered by [flashbots](https://github.com/flashbots)
# "mev-rs" - mev-boost, relays and builder are all spun up, powered by [mev-rs](https://github.com/ralexstokes/mev-rs/)
# We have seen instances of multibuilder instances failing to start mev-relay-api with non zero epochs
mev_type: null

# Parameters if MEV is used
mev_params:
  # The image to use for MEV boost relay
  mev_relay_image: flashbots/mev-boost-relay
  # The image to use for the builder
  mev_builder_image: ethpandaops/flashbots-builder:main
  # The image to use for the CL builder
  mev_builder_cl_image: sigp/lighthouse:latest
  # The image to use for mev-boost
  mev_boost_image: flashbots/mev-boost
  # Parameters for MEV Boost. This overrides all arguments of the mev-boost container
  mev_boost_args: []
  # Extra parameters to send to the API
  mev_relay_api_extra_args: []
  # Extra parameters to send to the housekeeper
  mev_relay_housekeeper_extra_args: []
  # Extra parameters to send to the website
  mev_relay_website_extra_args: []
  # Extra parameters to send to the builder
  mev_builder_extra_args: []
  # Prometheus additional configuration for the mev builder participant.
  # Execution, beacon and validator client targets on prometheus will include this configuration.
  mev_builder_prometheus_config:
    # Scrape interval to be used. Default to 15 seconds
    scrape_interval: 15s
    # Additional labels to be added. Default to empty
    labels: {}
  # Image to use for mev-flood
  mev_flood_image: flashbots/mev-flood
  # Extra parameters to send to mev-flood
  mev_flood_extra_args: []
  # Number of seconds between bundles for mev-flood
  mev_flood_seconds_per_bundle: 15
  # Optional parameters to send to the custom_flood script that sends reliable payloads
  custom_flood_params:
    interval_between_transactions: 1

# Enables Xatu Sentry for all participants
# Defaults to false
xatu_sentry_enabled: false

# Xatu Sentry params
xatu_sentry_params:
  # The image to use for Xatu Sentry
  xatu_sentry_image: ethpandaops/xatu:latest
  # GRPC Endpoint of Xatu Server to send events to
  xatu_server_addr: localhost:8080
  # Enables TLS to Xatu Server
  xatu_server_tls: false
  # Headers to add on to Xatu Server requests
  xatu_server_headers: {}
  # Beacon event stream topics to subscribe to
  beacon_subscriptions:
    - attestation
    - block
    - chain_reorg
    - finalized_checkpoint
    - head
    - voluntary_exit
    - contribution_and_proof
    - blob_sidecar

# Apache params
# Apache public port to port forward to local machine
# Default to port None, only set if apache additional service is activated
apache_port: null

# Global tolerations that will be passed to all containers (unless overridden by a more specific toleration)
# Only works with Kubernetes
# Example: tolerations:
# - key: "key"
#   operator: "Equal"
#   value: "value"
#   effect: "NoSchedule"
#   toleration_seconds: 3600
# Defaults to empty
global_tolerations: []

# Global node selector that will be passed to all containers (unless overridden by a more specific node selector)
# Only works with Kubernetes
# Example: global_node_selectors: { "disktype": "ssd" }
# Defaults to empty
global_node_selectors: {}

# Global parameters for keymanager api
# This will open up http ports to your validator services!
# Defaults to false
keymanager_enabled: false

# Global paarameter to set the exit ip address of services and public ports
port_publisher:
  # if you have a service that you want to expose on a specific interfact; set that IP here
  # if you set it to auto it gets the public ip from ident.me and sets it
  # Defaults to constants.PRIVATE_IP_ADDRESS_PLACEHOLDER
  # The default value just means its the IP address of the container in which the service is running
  nat_exit_ip: KURTOSIS_IP_ADDR_PLACEHOLDER
  # The start value gets used as a seed for TCP and UDP discovery ports for el/cl client
  # Defaults to None - no public ports
  public_port_start: null

Example configurations

Verkle configuration example
participants:
  - el_type: geth
    el_image: ethpandaops/geth:<VERKLE_IMAGE>
    elExtraParams:
    - "--override.verkle=<UNIXTIMESTAMP>"
    cl_type: lighthouse
    cl_image: sigp/lighthouse:latest
  - el_type: geth
    el_image: ethpandaops/geth:<VERKLE_IMAGE>
    elExtraParams:
    - "--override.verkle=<UNIXTIMESTAMP>"
    cl_type: lighthouse
    cl_image: sigp/lighthouse:latest
  - el_type: geth
    el_image: ethpandaops/geth:<VERKLE_IMAGE>
    elExtraParams:
    - "--override.verkle=<UNIXTIMESTAMP>"
    cl_type: lighthouse
    cl_image: sigp/lighthouse:latest
network_params:
  deneb_fork_epoch: 0
additional_services: []
wait_for_finalization: false
wait_for_verifications: false
global_log_level: info
A 3-node Ethereum network with "mock" MEV mode. Useful for testing mev-boost and the client implementations without adding the complexity of the relay. This can be enabled by a single config command and would deploy the [mock-builder](https://github.com/marioevz/mock-builder), instead of the relay infrastructure.
participants:
  - el_type: geth
    el_image: ''
    cl_type: lighthouse
    cl_image: ''
    count: 2
  - el_type: nethermind
    el_image: ''
    cl_type: teku
    cl_image: ''
    count: 1
  - el_type: besu
    el_image: ''
    cl_type: prysm
    cl_image: ''
    count: 2
mev_type: mock
additional_services: []
A 5-node Ethereum network with three different CL and EL client combinations and mev-boost infrastructure in "full" mode.
participants:
  - el_type: geth
    cl_type: lighthouse
    count: 2
  - el_type: nethermind
    cl_type: teku
  - el_type: besu
    cl_type: prysm
    count: 2
mev_type: flashbots
network_params:
  deneb_fork_epoch: 1
additional_services: []
A 2-node geth/lighthouse network with optional services (Grafana, Prometheus, transaction-spammer, EngineAPI snooper, and a testnet verifier)
participants:
  - el_type: geth
    cl_type: lighthouse
    count: 2
snooper_enabled: true

Beacon Node <> Validator Client compatibility

Lighthouse VC Prysm VC Teku VC Lodestar VC Nimbus VC
Lighthouse BN
Prysm BN
Teku BN
Lodestar BN
Nimbus BN
Grandine BN

Custom labels for Docker and Kubernetes

There are 4 custom labels that can be used to identify the nodes in the network. These labels are used to identify the nodes in the network and can be used to run chaos tests on specific nodes. An example for these labels are as follows:

Execution Layer (EL) nodes:

  "com.kurtosistech.custom.ethereum-package-client": "geth",
  "com.kurtosistech.custom.ethereum-package-client-image": "ethereum-client-go-latest",
  "com.kurtosistech.custom.ethereum-package-client-type": "execution",
  "com.kurtosistech.custom.ethereum-package-connected-client": "lighthouse",

Consensus Layer (CL) nodes - Beacon:

  "com.kurtosistech.custom.ethereum-package-client": "lighthouse",
  "com.kurtosistech.custom.ethereum-package-client-image": "sigp-lighthouse-latest",
  "com.kurtosistech.custom.ethereum-package-client-type": "beacon",
  "com.kurtosistech.custom.ethereum-package-connected-client": "geth",

Consensus Layer (CL) nodes - Validator:

  "com.kurtosistech.custom.ethereum-package-client": "lighthouse",
  "com.kurtosistech.custom.ethereum-package-client-image": "sigp-lighthouse-latest",
  "com.kurtosistech.custom.ethereum-package-client-type": "validator",
  "com.kurtosistech.custom.ethereum-package-connected-client": "geth",

ethereum-package-client describes which client is running on the node. ethereum-package-client-image describes the image that is used for the client. ethereum-package-client-type describes the type of client that is running on the node (execution,beacon or validator). ethereum-package-connected-client describes the CL/EL client that is connected to the EL/CL client.

Proposer Builder Separation (PBS) emulation

To spin up the network of Ethereum nodes with an external block building network (using Flashbot's mev-boost protocol), simply use:

kurtosis run github.com/ethpandaops/ethereum-package '{"mev_type": "full"}'

Starting your network up with "mev_type": "full" will instantiate and connect the following infrastructure to your network:

  1. Flashbot's block builder & CL validator + beacon - A modified Geth client that builds blocks. The CL validator and beacon clients are lighthouse clients configured to receive payloads from the relay.
  2. mev-relay-api - Services that provide APIs for (a) proposers, (b) block builders, (c) data
  3. mev-relay-website - A website to monitor payloads that have been delivered
  4. mev-relay-housekeeper - Updates known validators, proposer duties, and more in the background. Only a single instance of this should run.
  5. mev-boost - open-source middleware instantiated for each EL/Cl pair in the network, including the builder
  6. mev-flood - Deploys UniV2 smart contracts, provisions liquidity on UniV2 pairs, & sends a constant stream of UniV2 swap transactions to the network's public mempool.
Caveats when using "mev_type": "full"
  • Validators (64 per node by default, so 128 in the example in this guide) will get registered with the relay automatically after the 1st epoch. This registration process is simply a configuration addition to the mev-boost config - which Kurtosis will automatically take care of as part of the set up. This means that the mev-relay infrastructure only becomes aware of the existence of the validators after the 1st epoch.
  • After the 3rd epoch, the mev-relay service will begin to receive execution payloads (eth_sendPayload, which does not contain transaction content) from the mev-builder service (or mock-builder in mock-mev mode).
  • Validators will start to receive validated execution payload headers from the mev-relay service (via mev-boost) after the 4th epoch. The validator selects the most valuable header, signs the payload, and returns the signed header to the relay - effectively proposing the payload of transactions to be included in the soon-to-be-proposed block. Once the relay verifies the block proposer's signature, the relay will respond with the full execution payload body (incl. the transaction contents) for the validator to use when proposing a SignedBeaconBlock to the network.

This package also supports a "mev_type": "mock" mode that will only bring up:

  1. mock-builder - a server that listens for builder API directives and responds with payloads built using an execution client
  2. mev-boost - for every EL/CL pair launched

For more details, including a guide and architecture of the mev-boost infrastructure, go here.

Pre-funded accounts at Genesis

This package comes with 20 prefunded keys for testing.

Here's a table of where the keys are used

Account Index Component Used In Private Key Used Public Key Used Comment
0 Builder As coinbase
0 mev_custom_flood As the receiver of balance
1 blob_spammer As the sender of blobs
3 transaction_spammer To spam transactions with
4 goomy_blob As the sender of blobs
6 mev_flood As the contract owner
7 mev_flood As the user_key
8 assertoor As the funding for tests
11 mev_custom_flood As the sender of balance
12 l2_contracts Contract deployer address

Developing On This Package

First, install prerequisites:

  1. Install Kurtosis itself

Then, run the dev loop:

  1. Make your code changes

  2. Rebuild and re-run the package by running the following from the root of the repo:

    kurtosis run . "{}"

    NOTE 1: You can change the value of the second positional argument flag to pass in extra configuration to the package per the "Configuration" section above! NOTE 2: The second positional argument accepts JSON.

To get detailed information about the structure of the package, visit the architecture docs.

When you're happy with your changes:

  1. Create a PR
  2. Add one of the maintainers of the repo as a "Review Request":
    • parithosh (Ethereum Foundation)
    • barnabasbusa (Ethereum Foundation)
    • pk910 (Ethereum Foundation)
    • samcm (Ethereum Foundation)
    • h4ck3rk3y (Kurtosis)
    • mieubrisse (Kurtosis)
    • leederek (Kurtosis)
  3. Once everything works, merge!

ethereum-package's People

Contributors

0xtylerholmes avatar abhijeetbhagat avatar avalonche avatar barnabasbusa avatar bharath-123 avatar cbermudez97 avatar franjoespejo avatar github-actions[bot] avatar h4ck3rk3y avatar kamilchodola avatar leeederek avatar leoporoli avatar lyfsn avatar marioevz avatar megaredhand avatar mieubrisse avatar nflaig avatar pablocastellano avatar parithosh avatar peeeekay avatar piwonskp avatar pk910 avatar prestonvanloon avatar samcm avatar sauliusgrigaitis avatar savid avatar skylenet avatar tedim52 avatar victorcolombo avatar vuittont60 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ethereum-package's Issues

fix a bug in the eth2-package where it doesn't really parse arguments properly

I am guessing this is a bug from the proto -> no proto move

{
"participants": [
{
"el_client_type": "geth",
"el_client_image": "ethereum/client-go:v1.10.26",
"cl_client_type": "lighthouse",
"cl_client_image": "sigp/lighthouse:v3.3.0"
},
{
"el_client_type": "geth",
"el_client_image": "ethereum/client-go:v1.10.26",
"cl_client_type": "prysm",
"cl_client_image": "gcr.io/prysmaticlabs/prysm/beacon-chain:capella-post-merge,gcr.io/prysmaticlabs/prysm/validator:capella-post-merge"
},
{
"el_client_type": "geth",
"el_client_image": "ethereum/client-go:v1.10.26",
"cl_client_type": "prysm",
"cl_client_image": "gcr.io/prysmaticlabs/prysm/beacon-chain:capella-post-merge,gcr.io/prysmaticlabs/prysm/validator:capella-post-merge"
},
{
"el_client_type": "geth",
"el_client_image": "ethereum/client-go:v1.10.26",
"cl_client_type": "prysm",
"cl_client_image": "gcr.io/prysmaticlabs/prysm/beacon-chain:capella-post-merge,gcr.io/prysmaticlabs/prysm/validator:capella-post-merge"
}
],
"network_params": {
"network_id": "3151908",
"deposit_contract_address": "0x4242424242424242424242424242424242424242",
"seconds_per_slot": 12,
"slots_per_epoch": 32,
"num_validator_keys_per_node": 64,
"preregistered_validator_keys_mnemonic": "giant issue aisle success illegal bike spike question tent bar rely arctic volcano long crawl hungry vocal artwork sniff fantasy very lucky have athlete"
},
"wait_for_finalization": true,
"wait_for_verifications": false,
"global_client_log_level": "info"
}

is the input,

it spits out

image

@parithosh reported it. looking

Add beacon-metrics-gazer

Add beacon-metrics-gazer as a new tool. This requires a new docker image to be ran (beacon-metrics-gazer) + prometheus to target this docker image, and grafana to display the data.

Missing MEV productization

This is a ticket to self for things to do after / during gyani/full-mev is merged

  • make the mev_params passable
  • create a genesis context that has genesis time, different epochs(capella, bellatrix, genesis forK), and genesis_validators root and use those instead of the hardcoding

fix the Nimbus CL launcher by passing a valid `--network`

We currently seem to be passing path to a file for the network, it doesn't work.

Here are the logs

INF 2022-11-10 19:40:48.177+00:00 Launching beacon node                      topics="beacnde" version=v22.9.1-a84545-stateofus bls_backend=BLST cmdParams="@[\"--non-interactive=true\", \"--log-level=INFO\", \"--network=/genesis-data/output\", \"--data-dir=/root/consensus-data\", \"--web3-url=http://168.213.144.7:8551\", \"--nat=extip:168.213.144.11\", \"--enr-auto-update=false\", \"--rest\", \"--rest-address=0.0.0.0\", \"--rest-port=4000\", \"--validators-dir=/root/validator-keys\", \"--secrets-dir=/root/validator-secrets\", \"--doppelganger-detection=false\", \"--subscribe-all-subnets=true\", \"--num-threads=4\", \"--jwt-secret=/genesis-data/output/jwtsecret\", \"--metrics\", \"--metrics-address=0.0.0.0\", \"--metrics-port=8008\", \"--bootstrap-node=enr:-Ly4QH8b3iJKukfEPD1XDZoCFpmU7GQR9XI6kJgIqzdBhDDOV0cpPiCIYIlMx5JVignciuApPHn2W7eo8cVvRVUjc0YBh2F0dG5ldHOIAAAAAAAAAACEZXRoMpA5-P2IMAAAOP__________gmlkgnY0gmlwhKjVkAmJc2VjcDI1NmsxoQIVCMARYv9Q1j-6kq9L5MEiYe2qKvzDgWqekewj_e9keYhzeW5jbmV0cwCDdGNwgiMog3VkcIIjKA\"]" config="(configFile: None[InputFile], logLevel: \"INFO\", logStdout: auto, logFile: None[OutFile], eth2Network: Some(\"/genesis-data/output\"), dataDir: /root/consensus-data, validatorsDirFlag: Some(/root/validator-keys), secretsDirFlag: Some(/root/validator-secrets), walletsDirFlag: None[InputDir], eraDirFlag: None[InputDir], web3Urls: @[\"http://168.213.144.7:8551\"], web3ForcePolling: false, requireEngineAPI: None[bool], nonInteractive: true, netKeyFile: \"random\", netKeyInsecurePassword: false, agentString: \"nimbus\", subscribeAllSubnets: true, slashingDbKind: v2, numThreads: 4, jwtSecret: Some(\"/genesis-data/output/jwtsecret\"), cmd: noCommand, runAsServiceFlag: false, bootstrapNodes: @[\"enr:-Ly4QH8b3iJKukfEPD1XDZoCFpmU7GQR9XI6kJgIqzdBhDDOV0cpPiCIYIlMx5JVignciuApPHn2W7eo8cVvRVUjc0YBh2F0dG5ldHOIAAAAAAAAAACEZXRoMpA5-P2IMAAAOP__________gmlkgnY0gmlwhKjVkAmJc2VjcDI1NmsxoQIVCMARYv9Q1j-6kq9L5MEiYe2qKvzDgWqekewj_e9keYhzeW5jbmV0cwCDdGNwgiMog3VkcIIjKA\"], bootstrapNodesFile: , listenAddress: 0.0.0.0, tcpPort: 9000, udpPort: 9000, maxPeers: 160, hardMaxPeers: None[int], nat: (hasExtIp: true, extIp: 168.213.144.11), enrAutoUpdate: false, weakSubjectivityCheckpoint: None[Checkpoint], syncLightClient: false, trustedBlockRoot: None[Eth2Digest], finalizedCheckpointState: None[InputFile], finalizedCheckpointBlock: None[InputFile], nodeName: \"\", graffiti: None[GraffitiBytes], strictVerification: false, stopAtEpoch: 0, stopAtSyncedEpoch: 0, metricsEnabled: true, metricsAddress: 0.0.0.0, metricsPort: 8008, statusBarEnabled: true, statusBarContents: \"peers: $connected_peers;finalized: $finalized_root:$finalized_epoch;head: $head_root:$head_epoch:$head_epoch_slot;time: $epoch:$epoch_slot ($slot);sync: $sync_status|ETH: $attached_validators_balance\", rpcEnabled: None[bool], rpcPort: None[Port], rpcAddress: None[ValidIpAddress], restEnabled: true, restPort: 4000, restAddress: 0.0.0.0, restAllowedOrigin: None[TaintedString], restCacheSize: 3, restCacheTtl: 60, restRequestTimeout: 0, restMaxRequestBodySize: 16384, restMaxRequestHeadersSize: 64, keymanagerEnabled: false, keymanagerPort: 5052, keymanagerAddress: 127.0.0.1, keymanagerAllowedOrigin: None[TaintedString], keymanagerTokenFile: None[InputFile], lightClientDataServe: true, lightClientDataImportMode: only-new, lightClientDataMaxPeriods: None[uint64], inProcessValidators: true, debugForkChoice: false, discv5Enabled: true, dumpEnabled: false, directPeers: @[], doppelgangerDetection: false, syncHorizon: 50, terminalTotalDifficultyOverride: None[TaintedString], validatorMonitorAuto: false, validatorMonitorPubkeys: @[], validatorMonitorTotals: false, safeSlotsToImportOptimistically: None[uint16], suggestedFeeRecipient: None[Address], payloadBuilderEnable: false, payloadBuilderUrl: \"\")"
NOT 2022-11-10 19:40:48.177+00:00 Starting metrics HTTP server               topics="beacnde" url=http://0.0.0.0:8008/metrics
INF 2022-11-10 19:40:48.181+00:00 Threadpool started                         topics="beacnde" numThreads=4
NOT 2022-11-10 19:40:48.237+00:00 New database from snapshot                 topics="beacnde" genesisBlockRoot=25781dd2 genesisStateRoot=d9e0ff1b tailBlockRoot=25781dd2 tailStateRoot=d9e0ff1b fork="(previous_version: 10000038, current_version: 30000038, epoch: 0)" validators=128 tailStateSlot=0 genesisStateSlot=0
INF 2022-11-10 19:40:48.238+00:00 Loading block DAG from database            topics="beacnde" path=/root/consensus-data/db
ERR 2022-11-10 19:40:48.644+00:00 State from database does not match network, check --network parameter genesis=25781dd2:0 tail=25781dd2:0 headRef=25781dd2:0 stateFork="(previous_version: 10000038, current_version: 30000038, epoch: 0)" configFork="(previous_version: 20000038, current_version: 30000038, epoch: 0)"

How to reproduce?

Use the following YML to start the module

participants:
- elType: geth
  elImage: ethereum/client-go:v1.10.25
  clType: lighthouse
  clImage: sigp/lighthouse:v3.1.2
- elType: geth
  elImage: ethereum/client-go:v1.10.25
  clType: nimbus
  clImage: parithoshj/nimbus:merge-a845450
network:
  networkId: '3151908'
  depositContractAddress: '0x4242424242424242424242424242424242424242'
  secondsPerSlot: 12
  slotsPerEpoch: 32
  altairForkEpoch: 0
  mergeForkEpoch: 0
  totalTerminalDifficulty: 0
  numValidatorKeysPerNode: 64
  preregisteredValidatorKeysMnemonic: giant issue aisle success illegal bike spike
    question tent bar rely arctic volcano long crawl hungry vocal artwork sniff fantasy
    very lucky have athlete
waitForMining: false
waitForFinalization: true
waitForVerifications: true
verificationsEpochLimit: 5
logLevel: info

Error: An error occurred validating 'args'

Getting error after running command :

kurtosis run --enclave-id eth2 github.com/kurtosis-tech/eth2-package "$(cat /mnt/nvme/eth2-package/eth2-package-params.json)"

Im Edit Some files on src and static

Logs :

Error: An error occurred validating arg 'args'
 --- at /home/circleci/project/cli/cli/command_framework/lowlevel/lowlevel_kurtosis_command.go:290 (LowlevelKurtosisCommand.MustGetCobraCommand.func2) ---
Caused by: Error validating args, likely because it is not a valid JSON.
 --- at /home/circleci/project/cli/cli/commands/run/run.go:389 (validatePackageArgs) ---

Migrate this repo completely to the Starlark version

Requires:

  • Ensuring the Starlark version ( https://github.com/kurtosis-tech/eth2-module ) has complete parity with the old version, and works
  • Building the ability to execute Starlark modules directly from Github, without needing to clone them first
  • Showing @parithosh how to use the Starlark version
  • Migrating everything in the README to the new one
  • Switching the 'examples' repo to using the Starlark version ( kurtosis-tech/awesome-kurtosis#13 )
  • Waiting for a couple weeks, to shake out any issues
  • Deprecating this repo and pointing it to the Starlark one
  • Migrate all issues over to the new repo

Remove broken testnet-verifier

The testnet-verifier is currently broken for several reasons.

==
First, I run the module and almost immediately run:

./merge_testnet_verifier --ttd 0 --ttd-epoch-limit 0 --verif-epoch-limit 5 --client geth,http://<IP_GETH_NODE>:8545

and it runs the tests, even though we're not yet at epoch 5. I think it should wait for epoch 5 to be reached based on the description of verif-epoch-limit

==
Second, when I add a beacon node:

./merge_testnet_verifier --ttd 0 --ttd-epoch-limit 0 --verif-epoch-limit 5 --client geth,http://<IP_GETH_NODE>:8545 --client lighthouse,http://<IP_LIGHTHOUSE_BEACON>:4000

It loops forever with:

WARN[10-24|14:22:05] Error getting ttd block/slot             client=Lighthouse clientID=0 error="time before genesis"

When I manually kill it with Ctrl+C, before exiting it prints:

CRIT[10-24|14:50:40] Post-Merge Beacon Blocks Produced        client=Lighthouse clientID=0 pass=false extra="0 < 1"
CRIT[10-24|14:50:40] Post-Merge Justified Epochs              client=Lighthouse clientID=0 pass=false extra="0 < 1"
CRIT[10-24|14:50:40] Post-Merge Finalized Epochs              client=Lighthouse clientID=0 pass=false extra="0 < 2"
CRIT[10-24|14:50:40] Post-Merge Epoch Attestation Performance client=Lighthouse clientID=0 pass=false extra="0 < 85"
CRIT[10-24|14:50:40] Post-Merge Epoch Target Attestation Performance client=Lighthouse clientID=0 pass=false extra="0 < 85"
CRIT[10-24|14:50:40] Post-Merge Sync Participation Percentage client=Lighthouse clientID=0 pass=false extra="0 < 85"

All values are at zero.

==
Third, when I pass also a validator node:

./merge_testnet_verifier --ttd 0 --ttd-epoch-limit 0 --verif-epoch-limit 5 --client geth,http://<IP_GETH_NODE>:8545 --client lighthouse,http://<IP_LIGHTHOUSE_BEACON>:4000 --client lighthouse,http://<IP_LIGHTHOUSE_VALIDATOR>:5042

It immediately crashes complaining it can't reach the validator node.

==
Not entirely sure what's happening. Discussing with @parithosh on Discord, it could be that the verifier is confused because it's waiting for TTD to be reached, but here since we're running the network post merge from the start it's never really reached (and we set TTD to zero).
We should also check how the verifier is reading from the beacon and the validator, it seems it's having trouble retrieving the data.

bug: eth2-package has failing geth node

here was an error executing Starlark code
An error occurred executing instruction (number 34) at github.com/kurtosis-tech/eth2-package/src/participant_network/el/geth/geth_launcher.star[74:28]:
  add_service(name="el-client-0", config=ServiceConfig(image="ethereum/client-go:latest", ports={"engine-rpc": PortSpec(number=8551, transport_protocol="TCP", application_protocol=""), "rpc": PortSpec(number=8545, transport_protocol="TCP", application_protocol=""), "tcp-discovery": PortSpec(number=30303, transport_protocol="TCP", application_protocol=""), "udp-discovery": PortSpec(number=30303, transport_protocol="UDP", application_protocol=""), "ws": PortSpec(number=8546, transport_protocol="TCP", application_protocol="")}, files={"/genesis": "el-genesis-data", "/prefunded-keys": "geth-prefunded-keys"}, entrypoint=["sh", "-c"], cmd=["geth init --datadir=/execution-data /genesis/output/genesis.json && cp -r /prefunded-keys/* /execution-data/keystore/ && { for i in $(seq 1 6); do echo \"password\" >> /tmp/password.txt; done; } && geth --verbosity=3 --unlock=0x878705ba3f8Bc32FCf7F4CAa1A35E72AF65CF766,0x4E9A3d9D1cd2A2b2371b8b3F489aE72259886f1A,0xdF8466f277964Bb7a0FFD819403302C34DCD530A,0x5c613e39Fc0Ad91AfDA24587e6f52192d75FBA50,0x375ae6107f8cC4cF34842B71C6F746a362Ad8EAc,0x1F6298457C5d76270325B724Da5d1953923a6B88 --password=/tmp/password.txt --datadir=/execution-data --networkid=3151908 --http --http.addr=0.0.0.0 --http.vhosts=* --http.corsdomain=* --http.api=admin,engine,net,eth --ws --ws.addr=0.0.0.0 --ws.port=8546 --ws.api=engine,net,eth --ws.origins=* --allow-insecure-unlock --nat=extip:KURTOSIS_IP_ADDR_PLACEHOLDER --verbosity=3 --authrpc.port=8551 --authrpc.addr=0.0.0.0 --authrpc.vhosts=* --authrpc.jwtsecret=/genesis/output/jwtsecret --syncmode=full --bootnodes=\"\""], private_ip_address_placeholder="KURTOSIS_IP_ADDR_PLACEHOLDER"))
  Caused by: Unexpected error occurred starting service 'el-client-0'
  Caused by: An error occurred waiting for all TCP and UDP ports being open for service 'el-client-0' with private IP '10.2.0.5'; as the most common error is a wrong service configuration, here you can find the service logs:
  INFO [05-04|09:02:36.589] Maximum peer count                       ETH=50 LES=0 total=50
  INFO [05-04|09:02:36.590] Smartcard socket not found, disabling    err="stat /run/pcscd/pcscd.comm: no such file or directory"
  INFO [05-04|09:02:36.592] Set global gas cap                       cap=50,000,000
  INFO [05-04|09:02:36.592] Defaulting to pebble as the backing database
  INFO [05-04|09:02:36.592] Allocated cache and file handles         database=/execution-data/geth/chaindata cache=16.00MiB handles=16
  INFO [05-04|09:02:36.609] Opened ancient database                  database=/execution-data/geth/chaindata/ancient/chain readonly=false
  INFO [05-04|09:02:36.609] Writing custom genesis block
  INFO [05-04|09:02:36.616] Persisted trie from memory database      nodes=410 size=57.15KiB time="377.625µs" gcnodes=0 gcsize=0.00B gctime=0s livenodes=0 livesize=0.00B
  INFO [05-04|09:02:36.621] Successfully wrote genesis state         database=chaindata                      hash=482dd1..960441
  INFO [05-04|09:02:36.621] Defaulting to pebble as the backing database
  INFO [05-04|09:02:36.621] Allocated cache and file handles         database=/execution-data/geth/lightchaindata cache=16.00MiB handles=16
  INFO [05-04|09:02:36.640] Opened ancient database                  database=/execution-data/geth/lightchaindata/ancient/chain readonly=false
  INFO [05-04|09:02:36.640] Writing custom genesis block
  INFO [05-04|09:02:36.646] Persisted trie from memory database      nodes=410 size=57.15KiB time="348.084µs" gcnodes=0 gcsize=0.00B gctime=0s livenodes=0 livesize=0.00B
  INFO [05-04|09:02:36.649] Successfully wrote genesis state         database=lightchaindata                      hash=482dd1..960441
  INFO [05-04|09:02:36.693] Maximum peer count                       ETH=50 LES=0 total=50
  INFO [05-04|09:02:36.693] Smartcard socket not found, disabling    err="stat /run/pcscd/pcscd.comm: no such file or directory"
  INFO [05-04|09:02:36.696] Set global gas cap                       cap=50,000,000
  INFO [05-04|09:02:36.697] Allocated trie memory caches             clean=154.00MiB dirty=256.00MiB
  INFO [05-04|09:02:36.697] Using pebble as the backing database
  INFO [05-04|09:02:36.697] Allocated cache and file handles         database=/execution-data/geth/chaindata cache=512.00MiB handles=524,288
  INFO [05-04|09:02:36.718] Opened ancient database                  database=/execution-data/geth/chaindata/ancient/chain readonly=false
  Fatal: Failed to register the Ethereum service: ethash is only supported as a historical component of already merged networks
  Fatal: Failed to register the Ethereum service: ethash is only supported as a historical component of already merged networks
  Caused by: An error occurred while waiting for all TCP and UDP ports to be open
  Caused by: Unsuccessful ports check for IP '10.2.0.5' and port spec '{number:30303 transportProtocol:0 applicationProtocol:<nil> wait:0xc0002713d0}', even after '2' retries with '500' milliseconds in between retries. Timeout '15s' has been reached
  Caused by: An error occurred while calling network address '10.2.0.5:30303' with port protocol 'TCP' and using time out '14.49866325s'
  Caused by: dial tcp 10.2.0.5:30303: i/o timeout

Error encountered running Starlark code.

Error: An error occurred validating 'args'

Getting error after running command :

kurtosis run --enclave-id eth2 github.com/kurtosis-tech/eth2-package "$(cat /mnt/nvme/eth2-package/eth2-package-params.json)"

Im Edit Some files on src and static

Logs :

Error: An error occurred validating arg 'args'
 --- at /home/circleci/project/cli/cli/command_framework/lowlevel/lowlevel_kurtosis_command.go:290 (LowlevelKurtosisCommand.MustGetCobraCommand.func2) ---
Caused by: Error validating args, likely because it is not a valid JSON.
 --- at /home/circleci/project/cli/cli/commands/run/run.go:389 (validatePackageArgs) ---

Allow different tools to be enabled/disabled

instead of

{
  "participants": [
    {
      "el_client_type": "geth",
      "el_client_image": "ethpandaops/geth:dencun-devnet-8",
      "cl_client_type": "lighthouse",
      "cl_client_image": "ethpandaops/lighthouse:deneb-free-blobs",
      "count": 4
    }
  ],
  "network_params": {
    "capella_fork_epoch": 2,
    "deneb_fork_epoch": 4,
    "num_validator_keys_per_node": 77
  },
  "launch_additional_services": true,
  "wait_for_finalization": false,
  "wait_for_verifications": false,
  "global_client_log_level": "info"
}

have something like this:

{
  "participants": [
    {
      "el_client_type": "geth",
      "el_client_image": "ethpandaops/geth:dencun-devnet-8",
      "cl_client_type": "lighthouse",
      "cl_client_image": "ethpandaops/lighthouse:deneb-free-blobs",
      "count": 4
    }
  ],
  "network_params": {
    "capella_fork_epoch": 2,
    "deneb_fork_epoch": 4,
    "num_validator_keys_per_node": 77
  },
  "launch_additional_services": ['el-forkmon','tx_fuzz' ...]
  "wait_for_finalization": false,
  "wait_for_verifications": false,
  "global_client_log_level": "info"
}

Fix the Prysm beacon client which currently fails because fork versions don't match

time="2022-11-11 11:34:09" level=warning msg="The http-web3provider flag has been deprecated and will be removed in a future release,please use the execution endpoint flag instead execution-endpoint"
time="2022-11-11 11:34:09" level=info msg="Finished reading JWT secret from /genesis/output/jwtsecret"
time="2022-11-11 11:34:09" level=warning msg="Running on custom Ethereum network specified in a chain configuration yaml file" prefix=flags
time="2022-11-11 11:34:10" level=warning msg="Subscribing to All Attestation Subnets" prefix=flags
time="2022-11-11 11:34:10" level=warning msg="In order to receive transaction fees from proposing blocks, you must provide flag --suggested-fee-recipient with a valid ethereum address when starting your beacon node. Please see our documentation for more information on this requirement (https://docs.prylabs.network/docs/execution-node/fee-recipient)." prefix=node
time="2022-11-11 11:34:10" level=info msg="Checking DB" database-path="/consensus-data/beaconchaindata" prefix=node
time="2022-11-11 11:34:10" level=info msg="Opening Bolt DB at /consensus-data/beaconchaindata/beaconchain.db" prefix=db
time="2022-11-11 11:34:10" level=error msg="could not load genesis from file: loaded genesis fork version (0x30000038) does not match config genesis fork version (0x10000038)" prefix=main

Label the geth-client that is spun up alongside the builder differently from the other geth clients on the Ethereum nodes

Hey, when I run the eth2-package with full-mev, I expect there to be 2 Geth clients: 1 directly spun up with the MEV-builder, and another regular Geth client as part of an Ethereum full node.

In the output at the end (kurtosis enclave ls), it's not obvious to me as a user which one is which:

============================================== User Services ==============================================
UUID           Name                           Ports                                                  Status
e7f673086384   el-1-geth-lighthouse           engine-rpc: 8551/tcp -> 127.0.0.1:62861                RUNNING
                                              rpc: 8545/tcp -> 127.0.0.1:62863
                                              tcp-discovery: 30303/tcp -> 127.0.0.1:62862
                                              udp-discovery: 30303/udp -> 127.0.0.1:56858
                                              ws: 8546/tcp -> 127.0.0.1:62860
3048d9aafc12   el-2-geth-lighthouse           engine-rpc: 8551/tcp -> 127.0.0.1:62866                RUNNING
                                              rpc: 8545/tcp -> 127.0.0.1:62864
                                              tcp-discovery: 30303/tcp -> 127.0.0.1:62867
                                              udp-discovery: 30303/udp -> 127.0.0.1:62287
                                              ws: 8546/tcp -> 127.0.0.1:62865

count is not considered when doing the check for actual_num_validators

{
  "participants": [
    {
      "el_client_type": "geth",
      "el_client_image": "ethpandaops/geth:master",
      "cl_client_type": "teku",
      "cl_client_image": "consensys/teku:develop",
      "el_max_mem": 1000,
      "count": 10
    },
    {
      "el_client_type": "geth",
      "el_client_image": "ethpandaops/geth:master",
      "cl_client_type": "lighthouse",
      "cl_client_image": "sigp/lighthouse:latest",
      "el_max_mem": 1000,
      "count": 10
    },
    {
      "el_client_type": "geth",
      "el_client_image": "ethpandaops/geth:master",
      "cl_client_type": "lodestar",
      "cl_client_image": "chainsafe/lodestar:latest",
      "el_max_mem": 1000,
      "count": 10
    },
    {
      "el_client_type": "geth",
      "el_client_image": "ethpandaops/geth:master",
      "cl_client_type": "prysm",
      "cl_client_image": "prysmaticlabs/prysm-beacon-chain:latest,prysmaticlabs/prysm-validator:latest",
      "el_max_mem": 1000,
      "count": 10
    },
    {
      "el_client_type": "geth",
      "el_client_image": "ethpandaops/geth:master",
      "cl_client_type": "nimbus",
      "cl_client_image": "statusim/nimbus-eth2:amd64-latest",
      "el_max_mem": 1000,
      "count": 10
    }
  ],
  "network_params": {
    "network_id": "3151908",
    "deposit_contract_address": "0x4242424242424242424242424242424242424242",
    "seconds_per_slot": 12,
    "slots_per_epoch": 32,
    "genesis_delay": 1800,
    "capella_fork_epoch": 2,
    "deneb_fork_epoch": 1000,
    "num_validator_keys_per_node": 10,
    "preregistered_validator_keys_mnemonic": "giant issue aisle success illegal bike spike question tent bar rely arctic volcano long crawl hungry vocal artwork sniff fantasy very lucky have athlete"
  },
  "launch_additional_services": true,
  "wait_for_finalization": false,
  "wait_for_verifications": false,
  "verifications_epoch_limit": 5,
  "global_client_log_level": "info"
}
Evaluation error: fail: required_num_validtors - 64 is greater than actual_num_validators - 50
	at [github.com/kurtosis-tech/eth2-package/main.star:27:77]: run
	at [github.com/kurtosis-tech/eth2-package/src/package_io/parse_input.star:106:7]: parse_input
	at [0:0]: fail

Error encountered running Starlark code.```

Disable grafana login

Grafana login page is annoying. Grafana.ini should have:

[auth]
disable_login_form = true

[auth.anonymous]
enabled = true
org_role = Admin

Then it login should be disabled.

Report eth2-package v0.70

cannot running on latest version 0.70.0

error msg:

here was an error interpreting Starlark code 
Evaluation error: ExecRecipe: unexpected keyword argument "service_name"
	at [github.com/kurtosis-tech/eth2-package:33:88]: run
	at [github.com/kurtosis-tech/eth2-package/src/participant_network/participant_network.star:50:76]: launch_participant_network
	at [github.com/kurtosis-tech/eth2-package/src/participant_network/prelaunch_data_generator/cl_validator_keystores/cl_validator_keystore_generator.star:69:39]: generate_cl_validator_keystores
	at [0:0]: ExecRecipe

Error encountered running Starlark code.

Screenshot Running on v0.70.0 :
image

Screenshot Running on v0.68.0 :
image
image

And please update the package running on cloud vm :
image

Lodestar "beacon head slot" metric gets stuck around 31 if the network is run with only one participant

image

Kurtosis module executed with these params:
kurtosis module exec kurtosistech/eth2-merge-kurtosis-module --execute-params '{"participants": [{"elType": "geth","elImage": "parithoshj/geth:merge-371a525","clType": "lodestar"}],"network": {"numValidatorKeysPerNode": 64, "altairForkEpoch": 1, "mergeForkEpoch": 2},"waitForMining": false, "waitForFinalization": false,"logLevel": "debug"}'

eth2-with-one-lodestar-participant-logs.zip

This error doesn't happen if the network is executed with 2 participants (and one of them has a Lodestar CL) like this:

kurtosis module exec kurtosistech/eth2-merge-kurtosis-module --execute-params '{"participants": [{"elType": "geth","elImage": "parithoshj/geth:merge-371a525","clType": "teku","clImage": "parithoshj/teku:merge-b00acff","beaconExtraParams": ["--Xee-version kiln"]}, {"elType": "geth","elImage": "parithoshj/geth:merge-371a525","clType": "lodestar"}],"network": {"networkId": "3151908","depositContractAddress": "0x4242424242424242424242424242424242424242","secondsPerSlot": 12,"slotsPerEpoch": 32,"altairForkEpoch": 1,"mergeForkEpoch": 3,"totalTerminalDifficulty": 100000000,"numValidatorKeysPerNode": 64,"preregisteredValidatorKeysMnemonic": "giant issue aisle success illegal bike spike question tent bar rely arctic volcano long crawl hungry vocal artwork sniff fantasy very lucky have athlete"},"waitForMining": false,"waitForFinalization": false,"logLevel": "debug"}'

Resolve Geth block numbers getting stuck

It seems like there's an issue where, sporadically, Geth block numbers will get stuck at really low values (e.g. 3, 7, 9, etc.). It doesn't seem to be deterministic either. Maybe a Geth bug??

Simplify the genesis data generation

In this PR: ethpandaops/ethereum-genesis-generator#10, the ETH folks made a significant change to how genesis data generator works w.r.t. the config files it takes as input.
Before that, we had to provide the config YAML files all assembled (which is why we have this complex templating logic in the module).
Now, we just have to set the env variable defined in values.env, and the entrypoint.sh scripts takes care of doing the substitution and generating the genesis files.

Since we have a nice way to set env variable in docker container within Kurtosis, I believe we could find a way to completely get rid of the file templating logic we currently have. Plus, right now we're manually calling the python scripts to generate the genesis files. This could maybe go away as well.

To go incrementally, the simplest is maybe to just call the entrypoint.sh script manually at first (replace the python commands). Then if we want to go one level deeper, we can see how we could get the container to automatically generate the files when it starts (i.e. no overriding the entrypoint command as we do now)

dev experience enhancement - joining from outside - additional validator keys

Could be nice to have an additional parameter that allow to specify something like numExternalValidatorKeys: 64 so the prelaunch generator could generate additional keys and add them to genesis, so we could join the network from outside without going through the deposit flow.

we could then have a script that detect the p2p CL\EL peers by hitting the apis and build /ip4/127.0.0.1/tcp/62.../p2p/16Ui...aXRz and enode, so we can add them as static peers from outside.
Maybe this could be automated as well and included in the json output of the module.

config files and genesis can be picked from the generator docker volume.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.