• Create a basic DAO Program

• Make it upgradeable

• Link it with the Lido Program

• An instruction to invoke any of the DAO related instruction in the Lido Program

Here’s an idea to enforce keeping the stake pool balanced, and eliminate the need for rebalance entirely. Not sure if this is worth exploring, but I’m writing this down to get it out of my head.

Considerations

• We assume we have some function fn next_validator(&ValidatorList) -> ValidatorStakeInfo, that returns the next validator that we should deposit to. If we want to enforce a uniform balance, this function would return the validator with the minimal stake balance.
• We can’t run this function on delegate to determine at runtime which validator to delegate to, because it would require all validator accounts as inputs. But we can let the client compute this, and have Lido verify it. It would simply not allow delegations that make the pool more balanced.

There is still the problem that if the new amount to delegate is large in comparison to the current balances, a single delegation could move a validator from being the minimum to being the maximum, and the result would still be unbalanced. We can mitigate this by by enforcing an upper bound on the difference in stake balance between the minimal and maximal validator. For making a large deposit, the client would then have to break it up into multiple transactions.

For withdrawals we could do the opposite, only allow withdrawals from the validator with the maximum balance, and and enforce an upper bound on the stake difference between minimal and maximal. A large withdrawal would have to be broken up. (That would need to happen anyway if you want to withdraw more than the balance of a single validator.)

The “client” in these cases, is our maintenance bot/daemon, so it’s not a problem if it would need to break up some deposit, it does not affect user experience.

Conclusion

This approach puts a bound on the stake difference between validators, and it is enforced at every stake change Lido makes, so we never need to rebalance after stake changes though Lido. Only external changes (rewards or penalties) could change the balance, but Lido would always act towards restoring the balance, and if the accounts were balanced in the first place, validation rewards will not disturb that, so I don’t think that’s a problem in practice.

Alternative

Rather than enforcing this in the program, we could implement the validator decision only in the client. The client needs to compute that anyway, regardless of whether the program validates it. Then a uniform balance is not guaranteed by the program, but it will be uniform in practice. If we enforce it in the program, then we could open up these maintenance commands to be called by anybody; if we implement it in the client, then maintenance commands must remain guarded.

At the moment, the SPL stake pool program hard-codes its own id (poo1B9L9nR3CrcaziKVYVpRX6A9Y1LAXYasjjfCbApj) in instruction builder functions, so it is difficult to use the crate with a “self-hosted” version of the stake pool program. (Not that difficult though, I’m adding the SPL as a submodule in #90, it would be just one additional commit to our fork to replace the id.)

The stake pool program does not yet exist on mainnet-beta, but it does on the testnet, where its upgrade authority is
2WJLqUYKdkQrbkFz2xffwLFCNDVW2ej2BEaLQLFEKHDb. Assuming the upgrade authority is not going to be sealed initially on mainnet-beta, users would be trusting whoever the upgrade authority is, with their funds.

I think the easiest way forward to reduce the trust surface is if we upload our own version and set the upgrade authority to the same upgrade authority as that of Lido.

The accounts_struct macro only accepts const argument in the end, this might not be the case for some programs. We should make it accept these arguments anywhere.

• Deploy To Buffer

• Approve The Buffer (to upgrade the program)

• Add / Remove DAO Members

• [ ]

It would be nice if we could eventually expose some metrics and graph them, perhaps through a little daemon that queries the Solana RPC for the Lido state, and exposes it in Prometheus format.

A few metrics that would be nice to have are

• solido_deposited_lamports_total, sum of of all deposit ever. From this we can compute e.g. amount staked per day.
• solido_withdrawn_lamports_total, sum of all withdrawals ever.
• solido_deposits_total, monotonically increasing counter of the number of deposits since inception, so we can compute e.g. number of deposits per day.
• solido_withdrawals_total, monotonically increasing counter of the number of withdrawals since inception, so we can compute e.g. number of withdrawals per day.
• Together the above metrics also give us the average amount per deposit and withdrawal, but to gain more insight, we could use a histogram. Prometheus histograms include a total sum and a count, so this subsumes the above metrics. The tricky thing is that you have to decide on the histogram buckets up front.
• solido_fee_st_sol_total, the total fees paid since inception, possibly split out by post into treasury, insurance, validation fees, etc.

Aside from this we would want the current pool balance and such, but these we can just query from the Lido state already, we do not need to add counters to the state for this.

A note about why to track withdrawals and deposits separately: without this, if you see the pool balance increase by 10 SOL, you can’t distinguish between a single 10 SOL deposit, and a 1000 SOL withdrawal + a 1010 SOL deposit.

• Initialise and setup a StakePool Program

  • Add Validators

• Initialise and setup a Lido Program (blank/no instructions?)

• Point Lido Program to Stake Pool Program

• Setup Owners and Upgrade Authorities

• Create a Mint for LSOL
• Connect it to the Lido Program

At the moment, the CLI only accepts all of the addresses its needs as command-line flags. This can get very tedious and make the commands unwieldy. We can alleviate that by reading some addresses from a config file instead: many of the addresses don’t change over time (e.g. the Multisig program id, Solido program id, Multisig instance address, Solido instance address, etc.).

I propose to read these addresses from ~/.config/solido.toml if that file exists, and to make the command-line flags optional. If they are provided, they take precedence over the config file.

Aside from adding support for the config file, we should document it.

@malikankit this is not technically a blocker for the MVP, but I think the multisig participants will be very grateful if we add it, so I put this on MVP.

Summary

For Solido we need to run a “bot”: a program that periodically inspects the on-chain state, and potentially executes some transactions, like updating the stake pool balance after a new epoch starts, or putting deposited SOL in the stake pool.

Question for @joe-bowman, @kucharskim, @naomijub, @glottologist

What is the preferred way to set this up, and to make it easy to operate?

Keep in mind when answering, this repository is intended to become public.

Details

• The program needs to be able to sign on behalf of a funded Solana account (the cost should be low, just the transaction fees). Currently we support key files and we will add support for the Ledger hardware wallet as well (#107). Any preferences on how to securely access private keys?
• The program needs to interact with the chain through the Solana RPC. We could connect to a public RPC endpoint like http://api.mainnet-beta.solana.com/, but I think it would be better to connect to our own validator, so the program needs to be able to able to acces the RPC port of our validator (or possibly a secondary, but at least one we control).
• Aside from the RPC, which could be local or on a private network, the program does not need internet access.
• It will be a Rust program that needs a Rust toolchain to build. It does not need the Solana SDK to build. The output is a dynamically linked native Linux binary. It does link against some libraries that I’m not sure have stable ABIs, so I don't expect this binary to be portable across different distros.
• Simplest would be for it to log to stdout, it would be great if we could access that somewhere when it runs. (E.g. journalctl if it runs under systemd.)

Open questions

• Daemon or one-shot? We can make it a one-shot program, and run it periodically with e.g. a systemd timer. Or we could make it a deamon and have it loop internally. One advantage of a deamon is that it could expose Prometheus metrics, which can be used to monitor that it’s not stuck. This would also be a nice place to integrate #96, once we have those additional metrics.
• How to package, where to store the artifact? I don’t know if we have a standard approach for this, so I’ll just throw in some options:
  • 🤠 Compile locally, scp to server (if server and workstation are running the same distro).
  • Compile on GitHub actions, attach the binary as artifact so you can download it from GitHub.
  • Build a Debian/«insert-favorite-distro» package, push it to an internal package repository such as Freight.
  • Write a Nix expression, build it and cache the result on Cachix. This is great for reproducible builds too. Could be automated on CI.
  • Build a container image and push it to some registry.
• How to run it?
  • 🤠 loginctl enable-linger and then start the binary and background it.
  • Write a systemd unit. (Either one-shot with timer, or daemon.) Would we include it here, or would it be generated by config management anyway?
  • Container manager?

71e5e55#diff-29a45ad0794ff5c05c858271ce5751ddcbfed098aeae0e40324753c7202a400a

40ebfc6#diff-29a45ad0794ff5c05c858271ce5751ddcbfed098aeae0e40324753c7202a400a

This is an idea that was just mentioned during the meeting with the Solana team: property-test some invariants of Lido.

One great way of doing this is to encode every operation in a data structure, derive Arbitrary for it, and then run it in cargo-fuzz. This way you get coverage-driven property testing which is very powerful.

The thing I’m not sure about is how this would pan out in practice, because not all interactions happen in the Rust program itself, there are interactions with the Solana validator. Perhaps this thing that is also used in tests could be useful here.

At the moment, the solido CLI does not integrate with any software or hardware wallet; it reads a key pair from ~/.config/solana/id.json (or an alternative file if desired) and it uses that to sign. The associated account pays for the transaction fees and making accounts rent-exempt.

When we are going to set up the multisig, I suppose that we don’t want to use a key pair stored in a file. @malikankit , @mkaczanowski, do you know what we will use instead? The Solana CLI does support hardware wallets, so I think it should be possible to support this in solido as well.

Summary

Instead of having one stake account per validator, we need to have multiple.

Details

Currently StakeDeposit takes the validator to delegate to. It generates a derived address for that validator, and initializes that with a stake account. At some later point, when the stake is fully active, we can deposit that stake account into the stake pool. This means that:

• At most num_validators stake accounts exist.
• While stake is activating, these accounts can not be used for staking new deposits.
• Therefore we can do at most num_validators StakeDeposit calls per epoch, and fewer than that if stake takes longer than one epoch to activate.

If we need to be careful with how we spend those StakeDeposit calls, that will be problematic if we want to make this an “anybody can call” operation later, because somebody could call StakeDeposit at the start of the epoch, and any deposits that come in during the epoch can only be processed in the next epoch.

Proposal

• Instead of having one stake account per validator, we have an infinite number of them (or say 2⁶⁴), all program-derived, and indexed with a u64.
• Per validator we track a begin: u64 and end: u64. Every stake account in the range begin..end must be an initialized stake account.
• In StakeDeposit, if get_stake_account(end - 1) is in any state that is not 0% activated, then we need to initialize the next account at index end, and bump end.
• If get_stake_account(end - 1) is 0% activated, then we can temporarily create a new stake account at index end, and immediately merge it into account end - 1, because activating stake accounts can be merged.
• In DepositActiveStakeToPool, if get_stake_account(begin) is 100% activated, then we can deposit that stake account to the pool, and bump begin.

This has the following properties:

• end - begin increases by at most 1 per epoch. (If stake would never reach 100% activation, but we keep activating every epoch.)
• Even if stake takes longer than one epoch to activate, we can continue activating new stake in the current epoch.
• We can call StakeDeposit as often as we like. Because it creates a temporary stake account that is immediately merged, that stake will still start activating the next epoch, so even deposits that come in late in the epoch can be staked.

Test more than one deposit to test that lsol is correctly distributed:
One possible test goes by in the following steps:

• A deposits 200, gets 200 lsol
• Increment stake pool by 200 sol, new total = 400
• B deposits 100, should get 100*200/400 = 50, new total = 500

Suppose we don't have a cool-down period for withdrawals, and we return active stake accounts for withdrawals, that the user then needs to manually deactivate, as in #93. When there is lots of churn, this means we lose active stake, even if the amount of SOL managed does not change.

For Lido, too much churn is a problem. If it has X SOL staked, and in the epoch there is X SOL of withdrawals and X SOL of deposits. Then in the end the amount staked shouldn’t change, but all active accounts will be withdrawn, and all deposits still need to be activated, so in that epoch, Lido receives no rewards. The rewards all go to the withdrawers.

To put it differently, if withdraws return active stake, then the rewards earned in that epoch go to the withdrawer and not to Lido. If the churn is a large portion of the total balance, then Lido will miss out on many rewards.

I’m not sure how to solve this ... some ideas:

• Limit withdraws to e.g. a fraction of total stake per epoch? Then users can still withdraw eventually, but they can’t create too much churn.
• Charge a fee for withdrawals to compensate for the missed rewards? Maybe proportional to the block index in the epoch, so effectively a penalty for withdrawing near the end of an epoch?
• Have two withdrawals: withdrawing unstaked SOL from the reserve, and withdrawing active stake. The latter would only be allowed if the reserve is empty. This doesn’t fix the problem though, because you could first withdraw everything and then deposit everything.

When a user deposits SOL into Lido, she is entitled to stSOL.

The function that computes this is called calc_pool_tokens_for_deposit.

Let's rename it (and all references to) calc_st_sol_tokens_for_deposit.

Although the value is small, it's correct to remove the amount paid for the rent-free accounts when calculating values (e.g. deposit).

@FelixLutsch : Let me know if you have bandwidth to do this. If not, this can be re-assigned, but we should get the mockup ready this week.

• Review Tech Spec to understand 3 step withdrawal process (https://docs.google.com/document/d/1dVHVmQ_Yj22bJIqBhu3XDY5h6S-LdzQ1cVxCVq6_Zkw/edit#heading=h.c1eju6t2jniu)

• Mockup Withdrawal Screen

• Sync with Alex (Lido PM) and ask their designer to convert this into a finalised consistent design

• Pass this to Hritique for programming this

Lido Fees are to be distributed across the following 4 recipients -

a) Treasury
b) Insurance
c) Validators
d) Chorus One

For a), b) and d) , we will create an account that can receive stSOL tokens.

The owners for these 3 accounts respectively would be

a) Treasury - The Multisig PDA (?)
b) Insurance - The Multisig PDA (?)
d) Chorus One - Get One From Chorus (?)

• Confirm with @FelixLutsch
  • List of recipients and their owner addresses
  • Default Fee Split
  • Labels for each (treasury, insurance, developer(?))
  • Get an address for Chorus (testnet and mainnet)

[ ] Update FeeSpec data structure to reflect these labels

[ ] Update references to these labels in the Solido program

[ ] Add default values to the CLI / deployment documentation

Certain instructions are restricted to onlyManager.

For all such instructions, please confirm

1. accounts.manager has signed
2. accounts.manager == manager stored in the Lido state

Noticed that 1) is being checked for - but 2) was skipped in some instructions.

Note : Close issue if this has already been addressed in one of your open PRs.

Deposit Authority is one of the accounts params passed by the user in the DelegateDeposit instruction.

The current process_delegate_deposit function is missing a validation for the Deposit Authority passed by the user.

Let's add the check -
accounts.deposit_authority == deposit authority stored in the Lido state

We need a repository that contains instructions and scripts to enable multi-sig participants to perform

a) initial setup/deployment steps
b) recurring governance operations

Similar to https://github.com/lidofinance/terra-multisig-helper

As suggested by Ruud, we should pin LidoError enums:
"Given that these error codes are somewhat part of the public interface, I think it makes sense to pin them. That way we can also remove variants in the future without changing the error code of others."

An idea by Anatoly in the weekly meeting: it’s fine if withdrawals withdraw an activated stake account. That way the user immediately has custody of their funds, and it doesn’t make the user experience worse with respect to the earlier withdrawal note idea, because withdrawals are always going to be a two-step process due to the deactivation period. (Although now that I’m writing this down, with withdrawal notes you could have a bot make a final transfer, so it’s one step + wait for the user, assuming somebody else operates the bot.)

If withdrawals can return an activated stake account, that simplifies things tremendously for us, because we no longer have to track the deactivation process, we can return the stake account right from the stake pool to the user.

We will need deployment (and subsequent documentation) for

• deploying the Solido program on the blockchain (testnet and mainnet)
• CLI tool that validators / stakeholders will run to interact with the on-chain program
• Frontend for end-users to interact with Solido

A good first step would be for Fynn and Jason to catch up and sync on the steps (possibly updating the above list - if necessary). At some point, interfacing with @hritique on the frontend integration would be a good idea.

A good prototype for the documentation would be Lido for Ethereum README page.

PS : Jason's notes : Certus Wormhole project has a very good devnet setup, we could port the Solana part of it for Solido, it should make testing easier)

As per comments following: #133 (comment)

It would be good to know the differences, if any, on building project in container and locally using cargo build-bpf --remp-path-prefix. The BPF build doesn't respect RUSTFLAGS so that will need to be investigated first

As a : Developer of the Lido Program
I want : Stake Pool to have capabilities to restrict certain instructions to certain roles
So that : The Lido Program can wrap around the Stake Pool program in a way that restricts direct access to Stake Pool instructions

Eg.

• Restrict Deposits and Withdrawals to OnlyLidoProgram
• Restrict Add and Remove Validator to OnlyLidoProgram

To begin with, let's

• enable such gating

and then follow up with

• appropriate safety hatches

CLI arguments for governance functions using multisig should be default and not optional as we plan to deploy with multisig.
Remove Ooption from arguments, fix code and fix tests

Build inside container with bin diff

• Enlist all governance related functions to be added to Lido Program

• Enlist all governance related gating to be added to Lido Program

• Enlist of all governance related params to be added to Lido Program

• Get them peer-reviewed

• Implement

• Test with MultiSig

Deposit Instruction (for users to deposit SOL into the Lido Program)

• Basic Code
• 1 Test (Success)
• Check Incoming Parameters / Accounts
• Add Tests where this fails

• Set up a multi-sig contract that will act as Governance for Lido
• The contract (given sufficient signatures) should be able to call functions in the Lido program and the Stake Pool Program
• The contract should be able to upgrade the two programs
• The contract should act as owner - aka upgrade authority - of the Lido Program and Stake Pool Program
• Make this program upgreadable, i.e. given enough signatures, it can replace itself as the owner of the two programs.

• Enlist all governance related functions to be added to Lido Program

• Enlist all governance related gating to be added to Lido Program

• Enlist of all governance related params to be added to Lido Program

• Get them peer-reviewed

• Implement

• Test with MultiSig

Implement the TODO at program/tests/remove_validators.rs test_remove_validator_with_unclaimed_rewards.
Test removing a validator with some unclaimed rewards, this is dependent of increasing/decreasing the validator's stake account.

Background

At the moment, we expect the SOL balance of the stake pool to only go up through external modifications (so excluding deposits and withdrawals). At the moment these external modifications are only reward payments. However, at some point we may need to deal with penalties, and the SOL balance might go down.

Current approach

Currently, the stake pool performs a saturating subtract, which means it always computes a non-negative fee. If the SOL balance did in fact go down, no fees in stake pool tokens will be minted, but no negative rewards will be minted either. This means the loss is implicitly socialized over all stake pool token holders.

Proposal

In Lido we have an insurance account that part of the fees get paid into (when they are positive). At first I thought the stake pool should support negative fees, and then in Lido we could make a case distinction: credit to the fee accounts (treasury, insurance, etc.) if the fee was positive, but debit only from the insurance account when it was negative.

But aside from complications representing negative amounts as SPL tokens, this would not make up for the full loss, only for the fee percentage. For example, if the stake pool fee was configured to be 10%, then Lido could only compensate stSOL holders for 10% of the loss.

What we should probably do instead, is have a separate instruction for this case, where we burn tokens from the insurance fund equivalent in value to the lost SOL. (And if the insurance balance is insufficient, the remaining loss gets socialized.)

Lookup bump seeds from Lido state, instead of re-computing it everytime it's needed in the Lido program

Use docusaurus to create high level documentation for the solido project.

We need to create a script that maintainers can run on their infrastructure.

The script will periodically check for operations to be run (per epoch?) and run them if another maintainer hasn't already performed these operations (in this epoch?).

Let's also enlist these operations in our upcoming sync call.

Before we make the repository public, we should put a license in place and add copyright headers to the source files.

And in addition to being part of the pool, its stake status should be Active. We could make the instruction accept the index in the validator list where we expect to find the validator, or we could do a linear scan in the program itself.

Some error messages are not clear, improve them by saying what should be the correct value of checks.

Function that anyone can call to stake funds from the Lido Reserve Pool in the StakePool Program

• Basic Code

• 1 Test : Success with 1 Validator

• Checks

• More Tests

• Account for the 2 step process - see Slack conversation link and image below

As pointed out by Ruud:

IANAL, but if it’s a copy, or even an adaptation, we need to retain the copyright notice of the original code (SPL is Apache2):

You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works

But this is dealing with Lido, it’s not a copy of something in the Solana program library, is it?

Some of the Lido parts are copied/modified from the Stake Pool, We should take this into consideration when deciding which license to use for it

In the Lido program, there is an account that receives stake pool tokens on behalf of the Lido Program.

The owner of this account is the Deposit Authority PDA of the Lido Program (and is set the Deposit Authority of the Stake Pool on initialisation)

At some places, it is referred to as pool_token_to
At some places, it is referred to as stake_pool_token_holder

Let's go with stake_pool_token_holder - it's more clear.

• Change label pool_token_to with stake_pool_token_holder at all places

Also, whenever pool_token_to is passed in the accounts list, we are making multiple checks. We can just replace it with one check. (i.e. is it equal to the stake_pool_token_holder` stored in the Lido state)

• Ensure wherever it is passed in the accounts list - we match it with what's stored in the Lido state as stake_pool_token_holder

Calling a Solana instruction requires a list of accounts. These are essentially positional arguments, but on the Solana instruction level, not on the Rust level, so all we get in the code is a &[AccountInfo] which we have to construct and deconstruct manually.

Current Lido functions can take as many as 17 accounts, and the place where we deconstruct the list (in processor.rs) is far away from the place that constructs them (instruction.rs).

This makes me uneasy, it is very easy to accidentally have one address to too many, or to mix two up, and then the called function might do unexpected things. Now the function should be resilient against bad actors calling it with the wrong accounts anyway (so probably it needs to verify that all accounts are owned by Lido and such), but it would be nice to eliminate this class of problems at least for programs that use this code.

In anchor they mitigate this by defining a struct for the arguments (so they are named), and generating functions to convert to and from a list of AccountInfos. It looks like Anchor’s trait is integrated pretty tightly with the rest of the framework, so I think it would be hard to use without adopting the full thing, but maybe we can write a small proc macro that given a struct, generates functions to convert between that struct and &[AccountInfo]?

Any other ideas? @naomijub @enriquefynn @glottologist

A thought about the LidoError enum. Is it useful?

We go to great lengths to have members with docs for all failure cases, but who looks at the result? Only the CLI, or depositors that use the widget in the future. And those shouldn’t run into programming errors (like invalid preconditions). There’s no point in having enum variants for error cases that are not going to be handled programmatically anyway. I think we can replace most of them with a generic LidoError::Failed. What is very helpful though, is to print a message from the program just before it returns an error. That’s much more valuable for debugging than just the enum variant, and you can format additional details in there.

When I see the enumeration of instructions in the Stake Pool program - all the accounts to be provided in an instruction are well documented.

Eg. the comment from Lines 25-40 at https://github.com/solana-labs/solana-program-library/blob/b488aee06788a0d1cd0d407a42be054a5a1aa3cf/stake-pool/program/src/instruction.rs#L25

It would be useful to similarly document all instructions in instruction.rs of the Lido Program.

From #126
Pinning the Solana SDK and deps would be simpler if we could use a nixpkg rather than the installer script provided by the Solana team

As per: NixOS/nixpkgs#121009

The token will be called stSOL, we should be consistent in the code.