Description

Adding new validator keys to a Prysm validator with existing keys may not be straight-forward for some, so wanted to share what I learned during my experience importing them. The items below apply to anyone who closely followed the Prysm staking guide by Somer Esat.

Issue Overview

• In order to support a new deposit(s), the associated keys must be imported to the validator wallet directory which is located at /var/lib/prysm/validator.
• Because the user prysmvalidator owns the wallet directory, only it or root is able to successfully run the command needed to import new keys (it will fail due to lack of write permissions for other users).
• Because this user was created with /bin/false set, we cannot switch user to prysmvalidator to access the shell and run the import command.

Resolution

• One resolution here is to recursively (temporarily) change ownership of the the /var/lib/prysm/validator directory to be owned by your current user with the command sudo chown -R ryan:ryan /var/lib/prysm/validator (replacing ryan:ryan with your username).
• Next, run the import command cd /usr/local/bin then validator accounts import --keys-dir=/path/to/keystores where /path/to/keystores is the filepath to the directory containing all of (or just your new) keys.
• Go through the prompts and make sure you use the EXACT SAME wallet password as you did before.
• Import should succeed.
• Change ownership of wallet directory back to prysmvalidator user with sudo chown -R prysmvalidator:prysmvalidator /var/lib/prysm/validator
• Verify successful import with cd /usr/local/bin then sudo validator accounts list --wallet-dir=/var/lib/prysm/validator
• Restart your validator process with sudo systemctl restart prysmvalidator.

Additional

• If you experience the following error when running the import command, you will need to change the permissions on the keystore file using the command in the next step
  Fatal account: Could not import accounts: could not write accounts: file already exist without proper 0600
  • Command to change permissions sudo chmod 660 /var/lib/prysm/validator/direct/accounts/all-accounts.keystore.json
• You can perform all of these steps without stopping your validator. But you must restart your validator in order for it to read the new values in your wallet file.
• To confirm your new keys were picked up by the process, check the validator logs and ensure that you see a message along the lines of Waiting for deposit to be observed by beacon node.

In your lighthouse guide, you write the following when explaining the additional flags of the beacon node service:

--execution-endpoint=http://127.0.0.1:8551 The address of the Execution Client. Should be the same for all Execution Clients detailed in this guide.

I was wondering, is the = correct here? In the main config file, you don't use it.

[Unit]
Description=Lighthouse Consensus Client BN (Mainnet)
Wants=network-online.target
After=network-online.target[Service]
User=lighthousebeacon
Group=lighthousebeacon
Type=simple
Restart=always
RestartSec=5
ExecStart=/usr/local/bin/lighthouse bn \
  --network mainnet \
  --datadir /var/lib/lighthouse \
  --http \
  --execution-endpoint http://127.0.0.1:8551 \
  --execution-jwt /var/lib/jwtsecret/jwt.hex \
  --checkpoint-sync-url CheckpointSyncURL[Install]
WantedBy=multi-user.target

Great catch, thank you!

I have updated both the mainnet and testnet guides to remove the '=' from the flags description section. The config is correct.

Originally posted by @Obinna26 in #8 (comment)

In your lighthouse guide, you write the following when explaining the additional flags of the beacon node service:

--execution-endpoint=http://127.0.0.1:8551 The address of the Execution Client. Should be the same for all Execution Clients detailed in this guide.

I was wondering, is the = correct here? In the main config file, you don't use it.

[Unit]
Description=Lighthouse Consensus Client BN (Mainnet)
Wants=network-online.target
After=network-online.target[Service]
User=lighthousebeacon
Group=lighthousebeacon
Type=simple
Restart=always
RestartSec=5
ExecStart=/usr/local/bin/lighthouse bn \
  --network mainnet \
  --datadir /var/lib/lighthouse \
  --http \
  --execution-endpoint http://127.0.0.1:8551 \
  --execution-jwt /var/lib/jwtsecret/jwt.hex \
  --checkpoint-sync-url CheckpointSyncURL[Install]
WantedBy=multi-user.target

Hello, I'd just like to ask a question. I am looking for a guide for staking on the test net. I noticed that you have said that the Pyrmount guides have been depreciated. I was planning on using these guides to practice staking but now I am not sure. Do you have any advice or recommendations for a staking guide using a test net? I'd like to practice. Thanks!

P.S. I suppose that I am particularly referring to Ubuntu/Lighthouse guide's new note that was updated and committed here.

The current documentation for checking connected peers says to run the following command..

However, for Nimbus, we don't open the http connection and this actually should be geth attach ws://127.0.0.1:8546 -- notice that the ws:// instead of http:// and then 8546 instead of 8545

https://lighthouse-blog.sigmaprime.io/Quic.html

As of v4.5.0, lighthouse supports the quic protocol, which uses UDP and can be used between lighthouse clients for better latency and bandwidth as an alternative to the normal TCP 9000 port.

ufw settings would need to allow port 9001 for UDP

The flag --http-web3provider should be replaced with --execution-endpoint

Your guides are amazing, I was wondering if you were planning to do Prysm on Prater? I'm looking fwd to replace my Pyrmont client.

Hi and thanks for the great guides,

I propose to include links to alternative monitoring sites, such as https://www.rated.network/ or https://ethstakers.club. This change would encourage users to transition to these alternative platforms whenever one site encounters issues. Recent issues with incorrectly labeled missed attestations on beaconcha.in have highlighted the importance of diversifying the monitoring approach.

Hey @SomerEsat, I have found your guides very helpful!

An issue I have run into with Geth is that the proposed systemd configuration file uses the default timeout when shutting down. Since Geth sometimes can take quite a while to write its state to disk, this can lead to bad shutdowns (as systemd simply kills Geth after the default timeout) that corrupt Geth's state. This can often take hours or days to fix (I have for example recently seen it have to go back in history and start redownloading the state from over 1 week and 4 days ago).

Therefore, I think it might be helpful to add the TimeoutStopSec=600 variable in the Geth systemd configuration (i.e. a timeout of 10 minutes) so that Geth should almost certainly have enough time to write to disk before it is killed (as eluded to here for example: ethereum/go-ethereum#24416).

Thanks again for all you do for the space!

Following setup for Erigon as-is creates an error related to the --externalcl flag.

--externalcl without an argument will now cause erigon to halt (flag provided but not defined: -externalcl)

Removing the line entirely solves the issue as --externalcl is now default

Hi,

We are running a Besu+Teku node and Teku doesn't communicate with Besu unless --rpc-http-enabled flag is added to the configuration (it notices that EL is online but they cannot talk to each other).