our recent testruns in the aepp-sophia-examples repo where we execute a lot of chain rollbacks causes a crash, here the log:

• devmode-crash-after-snapshot.txt

would be great to have this issue addressed as soon as possible

At the moment, the plugin uses the configuration key custom_prefunded_accs_file to set the prefunded accounts file.

The newly added hard_forks configuration can instead be used to set the prefunded accounts file, in which case, the key custom_prefunded_accs_file can be removed from aeternity.

As I see, it currently uses base64 which is not used in the latest version of aeternity/aeserialization#19

I deactivated the rebar.config.script in order to take dependencies directly from the rebar.config and added this:

        {ebip39,           
                {git, "https://github.com/aeternity/ebip39.git"}}

It fetches ebip39 and compiles properly, however in runtime:

[error] CRASH REPORT Process <0.1657.0> with 0 neighbours exited with reason: call to undefined function ebip39:generate_mnemonic(128)

In my test app I can import and use it without an issue, what is different here @uwiger ?

Rollback and plugin status features aren't documented on readme file.

Currently, if reset chain to height 1 and do some transactions in sequence their hashes will be the same as before resetting. Because they produced from the same nonce and so. The only thing that changes is block/microblock hashes, due to they include a current time.

Having block hashes the same from run to run is useful in testing. I can reset the chain, run some script, record its output, and just compare it with a previous run. Also, it would help to avoid fluctuations in visual testing (when two screenshots are compared).

So, I propose adding an option to freeze the blocks' time to some constant value. If there are any other sources of randomness (a block nonce?) then freeze them as well.

Looks like it is also doable by modifying the node environment, but preferable to have it integrated, to simplify the setup
https://stackoverflow.com/a/41548434/6176994

Devmode

Marginal conditions

• Devtools need to be able to re-use existing APIs to submit transactions, etc.
• 1 node only setup (no peer connections expected)
  • no sync required, can be deactivated!
• Resource requirements should be decreased as much as possible. At least mining mustn't be required anymore for testing.

Configuration

These params can be included in a new section simulator of the YAML or provided in the command line when executing the binary:

Required

• active Used to activate the devmode with a custom consensus algorithm. Exposes the devmode management API if activated.
  • default: false
• block_interval Can be used to define a specific "mining interval" in seconds. By default blocks should be mined immediately.
  • default: 0 (immediately)
• protocol_version Define the protocol version to run the node with (hard_forks section should be omitted)
  • default: 5 (always the latest hardfork)
• start_time The unix timestamp to start with the first (or ongoing) block
  • default: the current unix timestamp
• derive_from_mnemonic Use a specific HD wallet mnemonic to generate initial accounts.
  • default: generated on-startup
• default_balance The default account balance to fund the generated accounts with, specified in AE.
  • default: 500
• total_accounts Number of accounts to generate at startup.
  • default: 10

Optional (to be discussed)

• db_path Some users prefer to set a custom db-path
• block_height_start The height to start the creation of blocks at. Only relevant if the node is started using a snapshot.
  • default: the following block based on the latest block of the snapshot
• impersonated_accounts An array of accounts addresses to be "impersonated". We probably need to investigate how to achieve that combined with tooling (e.g. SDK).
  • default: []

YAML example for the configuration:

- simulator:
  - active: true
  - chain:
    - db_path: '/opt/custom_dev_snapshot' # optional
    - protocol_version: 5
    - block_interval: 0
    - start_time: 15299999292
    - fork_from: 'https://mainnet.aeternity.io' # optional
    - block_height_start: 444450 # optional
  - accounts:
    - derive_from_mnemonic: <mnemonic_here>
    - default_balance: 500
    - total_accounts: 10
    - impersonated_accounts: ['ak_...', 'ak_...', 'ak_...'] # optional 

Comments

Protocol version

Here the question is how the node should behave if it is started with a snapshot of the chain. I think it should use the latest version of the snapshot and then immediately mine a new the protocol version defined in the config. If the protocol version provided is lower than the latest version in the snapshot the startup must fail.

Account funding

Regarding the account generation the question is if we want to keep the standard way like it is right now where we provide a genesis file that contains a JSON file with addresses and the corresponding amount to fund it. The above mentioned configuration options would be a bit more convenient for developers. In every case the "genesis" accounts and balances should be printed/logged. Maybe this doesn't matter at all if we decide to manage this via management API of the devmode. This way the tooling could call the API after startup and provide the addresses and respective amounts.

Block Height Start

Not sure if this is relevant. Does anybody see a reason to introduce this?

Database path

Not sure if this can currently already be set. But independent from that I think the devmode should - for simplicity - only support 1 specific db backend.

Fork from & impersonated accounts

This can be a very important feature if somebody want's to test functionality e.g. based on the current state of the mainnet. In best case we would be able to only fetch the top height from the remote node and be able continue the chain in the simulator.

On Ethereum there exist a lot of DeFi protocols like Compound or Aave which new projects might want to integrate into their decentralized applications. In some cases it is also required to impersonate other accounts - meaning that you need to be able to perform transactions on behalf of these accounts while not knowing the private key. I am currently not sure how tools & frameworks like Ganache or Hardhat are managing this. But they allow to broadcast transactions on behalf of these "impersonated" or "unlocked" accounts. We should definitely take a look into that.

See:

• https://hardhat.org/guides/mainnet-forking.html#customizing-hardhat-network-s-behavior
• https://www.youtube.com/watch?v=oBqS37ffNjE&t=292s

Start using environment variables

According to recent discussions the config options can also be provided via env-variables when executing the node's binary. So e.g. running ./aeternity start AE__DEVMODE__ACTIVE=true would start the devmode with default values. Basically there are different options to run the node with the desired configuration, see https://docs.aeternity.io/en/stable/configuration/#user-provided-configuration. @nikita-fuchs Raise your voice if we need sth. else here. But I think we already have enough options and just need to make sure it can easily be used with out tooling that wraps around it 😛

Devmode management API

This API should be active by default if node runs in devmode and can be a simple http API provided in a new group/tag.

Required

Update protocol version (trigger hardfork)

We need an endpoint that can be used to trigger a hardfork on-demand.

• PUT /devmode/protocol-version
  • Request payload example: {"value": 2}

Should fail if the protocol version doesn't exist and if the protocol version is equal or lower than the current version.

Jump into the future by block or time

We need an endpoint to be able to jump into the future to test block-height or time-based dependent logic.

By block

• PUT /devmode/block-height
  • Request payload example: {"type": "delta", "value": 20} (here we can distinguish between delta and actual block as already done for TTL)

Should fail if an actual block is provided which is lower than the current height.

By time

• PUT /devmode/time
  • Request payload example {"value": 86400} (amount of seconds to move forward)

Optional (to be discussed)

Fund accounts

It might be necessary to fund certain accounts on-demand where one would not want to rely on performing a SpendTx on behalf of an existing account. Not sure if it makes sense as we are able to fund accounts on startup already. Can probably be omitted.

Snapshot the chain state & revert to a specific chain state

This is a feature that Ganache provides. Currently wondering about the usecase. When would we need to make a snapshot during tests where we could revert back to at any point of time?

Probably the easiest way is just to allow the dev to jump back to a certain block height of the current chain by invoking the management API of the simulator.

References

For Ethereum there exists Ganache where we can take a look at and pick what we need:

• https://github.com/trufflesuite/ganache-core#features
• https://github.com/trufflesuite/ganache-cli#options-1
• https://github.com/trufflesuite/ganache-cli#management-api-methods
• https://github.com/trufflesuite/ganache-cli#custom-methods

Current State

• Config
  • chain
    • keyblock interval
    • microblock interval
    • auto-emission of microblocks
    • start_time - unix timestamp to start with the first (or ongoing) block
      • default: current unix timestamp
    • protocol_version - protocol version to run the node with (hard_forks section should be omitted)
      • default: always the latest hardfork
    • block_height_start - height to start the creation of blocks at
      • only relevant if the node is started using a snapshot.
      • default: following block based on the latest block of the snapshot
  • accounts
    • derive_from_mnemonic - mnemonic seed phrase to derive accounts from (@nikita-fuchs working on that afaik)
      • default: some seed phrase we always use OR a generated one (not sure what's better @nikita-fuchs)
    • default_balance - the default balance of all generated accounts
      • default: 500 Æ
    • total_accounts - the total amount of accounts to derive from mnemonic
      • default: 10
• API
  • chain
    • /emit_mb - Emit a microblock
    • /emit_kb?n=<N> - Emit N keyblocks
    • /kb_interval?secs=<S> - Set a keyblock interval of S seconds (secs=0 turns off)
    • /mb_interval?secs=<S> - Set a microblock interval of S seconds (secs=0 turns off)
    • /auto_emit_mb?auto_emit=on - Turn on auto-emission of microblocks on tx push
    • /auto_emit_mb?auto_emit=off - Turn off auto-emission of microblocks on tx push
    • time?secs=<S> - Jump S seconds into the future
    • protocol_version=<V> Jump to the (future) protocol version V

Other considerations

• did somebody ever test running devmode based on a mainnet/testnet snapshot? @nikita-fuchs @thepiwo
• we recently also discussed if we can fund a new account after we run in devmode based on a mainnet/testnet snapshop. it should already be doable but somebody needs to test @nikita-fuchs @thepiwo
• impersonating accounts when running devmode based on a mainnet/testnet snapshot -> how can we get control over an account where we do not know the private key? @uwiger @velzevur @hanssv

Originally posted by @marc0olo in #19 (comment)

I was debugging aeplugin_dev_mode_emitter.erl and found out that it actually was copied to the node in aeternity/aeternity#4055

As I see in this repo remains some unused functions (like top_changed handler and restart_block_timer) and would be less confusing to remove them 🙂

This is necessary in order to handle maintenance mode activation and de-activation.

/emit_kb?n= with high numbers fails with status 500, didn't see anything in logs