perun-network / erdstall Goto Github PK

Due to the operator now sending a client-config over the wire upon a client establishes a connection, we have to adapt the go-client to handle that case.

The client currently polls the operator for fresh deposit and balance proofs. It would be better to either

1. use a subscription instead or
2. make the call blocking, like the enclave call already is.

The issue with 2. might be that it only works on a very stable and uninterrupted connection between client and op. But it would be more straight forward to change the current implementation to that.

Currently, when a client connects to the operator he receives an old depositProof even tho he already exited and withdrew the system. This would not be as bad if the client had persistence, but even then would be inconsistent between uses on different PCs.

A client does not know if the received depositProof belongs to an ongoing session or a "finished" one.

BalanceProofs are not being sent, as expected.

Add support for an asset abstraction to the smart contract and then allow the system to handle multiple currencies at once.
We need this if we want to use Erdstall as a showcase for exchanges or other defi applications.

Implement a complete GrapheneOS deployment solution.

It should deploy a GrapheneOS to a local IntelSGX (or vm) inside which the prototype enclave runs as a remote server. Then it should connect a local remote enclave client and make it usable as a tee.Enclave.

Ideally, we extend the operator with a cmdline flag --tee with the possible values

• prototype (default): use local prototype enclave or
• grapheneos: start GrapheneOS Enclave server in local IntelSGX and connect to it with a remote enclave server.

Has the additional benefit of determining revert reasons.

The operator should forward incoming txs to the recipient, so that their client software can display an incoming pending tx.

We need to add a new subscription message type for this.

Related https://github.com/perun-network/erdstall-web3-client/issues/14

Right now, at least one participant has to call freeze once the freezing phase starts. If there are multiple clients that want to freeze the contract, their client impls would probably all call freeze at the same time, resulting in many reverting and unnecessary transactions.

When a sendTx message is received, it is handled in a go routine as to not block the message handler loop.

However, this could lead to an accumulation of go routines if many thousands txs are cached. So it would be better to just cache the requests and then send the replies once the new epoch starts.

Protocol enhancement

For this, the client/test Client needs to be extended to also support sending exit and withdrawal transactions.

Can then also use this in #39

The operator is currently unable to handle more than one challenge at once.
The two parallel handleChallenge calls race and one of the transactions can end up with a wrong account nonce resulting in this error:
ERRO[0070] Operator.handleChallenges: Failed to handle challenged event ChallengedEvent{Account: 0x22d491Bde2303f2f43325b2108D26f1eAbA1e32b, Epoch: 4}: sending challenge response: the tx doesn't have the correct nonce. account has nonce of: 3 tx has nonce of: 2

To enable parallel dispute testing, replace testChallenge(t, clients[0]) with testChallenge(t, clients...) in integration_test.go:76 and run the integration test.

Taking some standard CI config from some of our public Go projects, including

• tests
• vetting

The operator tracks the status of challenge response transactions and logs an error if a transaction fails. Currently we do not log the failure reason because it is unclear how failure information can be extracted from go-ethereum.

Ideally we would like to log an error with the error reason (e.g., "out of gas") if a challenge response transaction fails.

should be require(epoch > exitEpoch(), "withdraw: too early"); or better require(epoch > exitEpoch(), "withdraw epoch must be greater exit epoch");

The Operator tests currently use their own copy of a testing client. We should streamline this to remove the copy and use a common testing client.

Currently, the operator forwards incoming transactions straight to the enclave. This is unstable near phase shifts, where the operator might receive TXs for the next epoch already. We therefore need to enable tx caching for incoming transactions. Once a phase shift is performed, all cached transactions for this epoch should be pushed into the enclave.

The operator should create a subscription to challenges and immediately answer to them.

The client should have a new challenge cmd which sends a challenge on-chain. Note that opening a challenge automatically exits the user from the system once the operator sends the proof to the contract (answering to challenges happens via the same exit call). So the challenge cmd should not only call the challenge function on the contract, but then also subscribe to a corresponding Exiting event. If the event arrives, wait for the end of the epoch and then call withdraw. So this is the same logic as for normal exits, so code duplication can be avoided.

I think for now we don't need to let the client automatically send challenges if they don't receive a proof in time, as this is just a demo. We can do it in a later issue if we receive interest in this.

In #77, reorg handling is introduced into the block subscription and disabled in the prototype enclave. This is fine for a demo, but unsafe in a real setting. It needs to be handled in the enclave so that it can track the mined blocks itself.

#48 (comment)_

Currently, the operator starts a go routine for each challenge response transaction to track the mining status. This is not efficient and may fail if there is a large number of challenges coming in, as the number of concurrent go routines is limited.

A better method for transaction tracking would be to have a dedicated go routine that gets passed the transactions to be tracked, e.g., via a channel.

When a user exited the system successfully and then joins the system again, the RPCOperator still has buffered depositProofs, which are being sent to the client. It would be cleaner if, upon leaving the system, the operator would get rid of the cached proofs.

On client side this can be easily mitigated by checking the current DepositEpoch w.r.t. the latest received depositProof. This way the client can distinguish, whether or not to use this proof to show off-chain balances.

The Go CLI client is nice, but not suitable for all demonstrations. So we should also develop a browser-based client that can use Metamask as a web3 provider.

Create a Web3 Erdstall Client. Should be a working implementation which allows to connect e.g. MetaMask and use funds for transfers.

logrus should be used directly.

While testing the Operator on-chain, i noticed that the Enclave uses account index 0 and the Operator index 1.
When entering the mnemonic from a wallet that does not support multiple HD derivations (like metamask), the operator tries to do on-chain calls with the unfunded account.

It would therefore be easier to use the index 0 account of the entered mnemonic for on-chain calls.
This would only exchange 2 chars in the default configuration file and make testing easier.

While doing this, printing th OP and Enclave PK would also be good.

Bug  In tee/rpc.Enclave, if a tee.ErrEnclaveStopped is returned via an RPC call, then the receiver does not currently convert it back to tee.ErrEnclaveStopped. This means that calling code can no longer distinguish between error codes.
Fix  Create a function that converts errors into tee.ErrEnclaveStopped if they have textual equality. Apply this function to all returned errors.

We should let the operator forward incoming transaction to a recipient.
For this we add another subscription topic to the client subscription and just send any incoming transaction unchanged to the receiver.

This message fulfills no protocol purpose but is an informative message for clients. Until now, tx recipients only receive new balance proofs but don't know what txs aggregated to this new balance.

The operator can panic since gorilla does not allow for repeated reads on websocket connections that returned an error on a read call, resulting in this:

panic: repeated read on failed websocket connection

goroutine 49 [running]:
github.com/gorilla/websocket.(*Conn).NextReader(0xc00018fa20, 0xc000287af0, 0x470d9f, 0xecf258, 0xc000088190, 0x1313)
        /home/erdop/go/pkg/mod/github.com/gorilla/[email protected]/conn.go:965 +0x347
github.com/gorilla/websocket.(*Conn).ReadMessage(0xc00018fa20, 0xc000000002, 0xc000287bd8, 0x1, 0x1, 0xc0004897a0, 0x15e36d0)
        /home/erdop/go/pkg/mod/github.com/gorilla/[email protected]/conn.go:1026 +0x2f
github.com/perun-network/erdstall/operator.(*Peer).readMessages(0xc000485080)
        /home/erdop/erdstall/operator/rpc.go:156 +0x55
created by github.com/perun-network/erdstall/operator.(*RPCServer).connectionHandler
        /home/erdop/erdstall/operator/rpc.go:147 +0x427

Fixing this would mean to just remove the peer on the first read error, since gorilla does not seem to able to recover from it.
Affects operator/rpc.go Peer.readMessages.