shutter-network / rolling-shutter Goto Github PK

When a new key or cipher batch is received, check if we have now both of them and if so create a (placeholder) signature. Store it in the database and broadcast it.

• connects to a number of libp2p gossip networks
• logs data it receives
• provides function to publish data

• users should be able to validate that it's legit
• users should be able to get it even if they're not online while it's generated

Whenever a key, cipher batch, or signature is received, store it in the database if it's not there already. (Use placeholder db for now, so don't store real data)

Right now we have functions to handle inputs and outputs produced but nothing feeds it and reads from it:
https://github.com/shutter-network/rolling-shutter/blob/main/rolling-shutter/decryptor/decryptor.go

The keyper should listen for epoch key shares of their peers, validate them, and store them in their db. When it has enough, it should aggregate them, store the resulting decryption key in their db, and send a corresponding decryption key message to the decryptors.

in shlib/shcrypto

This is needed

• to store secret and public keys in config files and
• to send around signatures and store them in the database

• In the keyper, when decryption trigger is received, generate and broadcast epoch decryption key share
• only do it once (even if multiple decryption triggers for same epoch are received)
• share should be stored in db

• based on https://pkg.go.dev/github.com/ethereum/[email protected]/crypto/bn256/cloudflare
• part of https://github.com/shutter-network/shutter/tree/main/shlib/shcrypto
• public keys in G1 (for cheap aggregation on Ethereum), signatures in G2
• sign(sk, msg) -> sig
• aggregateSigs(sigs) -> sig
• aggregatePKs(pks) -> pk
• verify(sig, msg, pk) -> bool

• what data needs to be stored?
• what data needs to be persisted?
• how will it be accessed (by key, complicated queries, etc)?
• how should it be stored (relational database, key value store)?

check that they key matches the eon public key for the given epoch

register the validator for both the decryptor and the keyper

c.f. decryption key input handler which already does validation

• read connection details from config
• close database if something crashes
• crash if database is not initialized (for now: check if one particular table exists)

c.f. https://github.com/shutter-network/rolling-shutter/blob/main/rolling-shutter/keyper/kprdb/schema.sql#L47-L52

store:

• keyper ethereum addresses (use encode/decodeAddress from shdb package)
• threshold of set
• start epoch index of set

Store keyper identities in table separate from table for sets.

Right now, the p2p thing connects to all peers on startup. If a single connection fails, it exits. If over time peers disconnect, we might end up with too few peers.

Instead, on startup we should create a dedicated goroutine responsible for peer management. The goroutine regularly checks if we have too few connections. If so it tries to connect to more peers (using addresses from the config file). If there is an error (e.g., because the peer is offline), it doesn't quit but logs the error. The goroutine should be part of the error group already used for the topic subscriptions so that it's automatically stopped.

The goroutine should also regularly log the number of peers it is connected to.

Later, we can improve by syncing peer lists from other peers.

To create BLS signatures in G2, we need to first map the message to a point in G2. At the moment, we do this by hashing the message to a string, interpreting it as an integer, and multiplying it by the base point. This is not the proper way to do it.

• The cloudflare library has an issue for this: cloudflare/bn256#17
• If EIP-2537 is activated, we should probably switch to the BLS12-381 curve and we can use whatever the Eth2 clients use.
• If we have to implement it ourselves, we have two options:
  • Implement a proper algorithm, e.g., described here: https://datatracker.ietf.org/doc/draft-irtf-cfrg-hash-to-curve/
  • Implement "hash and check" which supposedly is insecure, but should be acceptable for our use case (because the attacker cannot choose the message that's signed) -- see here for a description, but in the context of BLS12-381.

new name: rolling-shutter

I ran three keypers and let them generate a eon public key. Then I manually inserted the key into the dbs of three decryptors and started them too. Finally, I ran the mocknode. Two of the three keypers paniced with the following message:

2021/10/15 18:09:11.657826 inputhandling.go:57: attempted to store multiple keys for same epoch 29
2021/10/15 18:09:11.681066 inputhandling.go:57: attempted to store multiple keys for same epoch 29
2021/10/15 18:09:11.684321 inputhandling.go:215: decrypting batch for epoch 29
panic: could not verify aggregated signature for epochID 29

goroutine 41 [running]:
github.com/shutter-network/shutter/shuttermint/decryptor.handleSignatureInput({0x176a3f0, 0xc000dd1a00}, {{0x178ecc0, 0xc000030990}, {0xc000dc6c40, 0x7, 0x7}, {0xc000121320, 0x1a}, 0xc000deaf60, ...}, ...)
	/home/jannik/Projects/Brainbot/rolling-shutter/rolling-shutter/decryptor/inputhandling.go:167 +0x99f
github.com/shutter-network/shutter/shuttermint/decryptor.handleEpoch({0x176a3f0, 0xc000dd1a00}, {{0x178ecc0, 0xc000030990}, {0xc000dc6c40, 0x7, 0x7}, {0xc000121320, 0x1a}, 0xc000deaf60, ...}, ...)
	/home/jannik/Projects/Brainbot/rolling-shutter/rolling-shutter/decryptor/inputhandling.go:234 +0x5be
github.com/shutter-network/shutter/shuttermint/decryptor.handleCipherBatchInput({0x176a3f0, 0xc000dd1a00}, {{0x178ecc0, 0xc000030990}, {0xc000dc6c40, 0x7, 0x7}, {0xc000121320, 0x1a}, 0xc000deaf60, ...}, ...)
	/home/jannik/Projects/Brainbot/rolling-shutter/rolling-shutter/decryptor/inputhandling.go:80 +0x296
github.com/shutter-network/shutter/shuttermint/decryptor.(*Decryptor).handleMessage(0xc00023e880, {0x176a3f0, 0xc000dd1a00}, 0xc000f9c300)
	/home/jannik/Projects/Brainbot/rolling-shutter/rolling-shutter/decryptor/decryptor.go:120 +0x375
github.com/shutter-network/shutter/shuttermint/decryptor.(*Decryptor).handleMessages(0xc00023e880, {0x176a3f0, 0xc000dd1a00})
	/home/jannik/Projects/Brainbot/rolling-shutter/rolling-shutter/decryptor/decryptor.go:95 +0x10a
github.com/shutter-network/shutter/shuttermint/decryptor.(*Decryptor).Run.func1()
	/home/jannik/Projects/Brainbot/rolling-shutter/rolling-shutter/decryptor/decryptor.go:77 +0x25
golang.org/x/sync/errgroup.(*Group).Go.func1()
	/home/jannik/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:57 +0x67
created by golang.org/x/sync/errgroup.(*Group).Go
	/home/jannik/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:54 +0x92
Error while executing task: d

right now we cast them from uint64 to int64 which loses precision

Tables:

• keys:
  • round id (uint64)
  • key placeholder (bytes32)
• cipher batches
  • round id (uint64)
  • cipher batch placeholder (bytes32)
• signature shares
  • round id (uint64)
  • signer index (uint64)
  • signature placeholder (bytes32)

Set this up in initdb subcommand

• right now there all 0
• we should read the config file
• throw away all messages with wrong instance id

Keypers should listen for decryption key shares of their peers and when they have enough of them aggregate and broadcast them.

assume it will be notified whenever new cipher batches or decryption keys are available

decrypt batch, sign the result, send it to p2p layer

Decryptors already send it to the decryption signatures topic. Create a new topic (e.g. aggregatedDecryptionSignature) and send it there.

Decryptors should also connect to this topic and validate messages.

• channel names are hardcoded
• keyper and decryptor use same p2p thing
• connect on startup, listen to messages, but ignore them

message types, only for gossiping:

• epoch end trigger
• epoch decryption key
• cipher batch
• signature
• aggregated signature

Config file is read at startup. The p2p layer of the application should connect to these addresses. The addresses should use libp2p multiaddr format. The config file should be validated. Format: TOML. Read it with viper. c.f. shuttermint config file validation.

At the moment, sending a new aggregated decryption signature is triggered by receiving a new unaggregated signature (and we have enough of them in the db). This means we send many unnecessary ones.

• store aggregated signatures in db table (one per hash)
• only broadcast signature when none is in db already

When decryption key and cipher batch are received, decrypt.

We already send a fake signature (see #19). Update this to send a proper one.

The decryptors need to know the eon keys so that they can validate that the epoch keys. Figure out how they get them and how they make sure they're valid/produced by the keypers.

Write a tool that

• connects to keypers and/or decryptors
• sends decryption triggers, decryption keys, cipher batches, etc.
• logs messages it receives

The goal is to be able to easily test the decryptor and the keyper node, without spinning up a whole network.

The mocknode should

• remember which transactions it encrypted and sent
• check that decryption signatures it receives are attesting to the right value

To check the signature, they'll need to know who signed it. Since we don't handle signer indices yet, nor have a list of decryptors yet, it's fine to put a single decryptor identity in the config file and check against that.

if we receive a decryption signature, we should verify that it was signed by one of the decryptors and ignore it if it wasn't

when running the keyper command, it should

• connect to a shuttermint chain (url from config)
• participate in DKG process

Eon key generation only

• parse them and the included data types
• convert them to a struct (one per message type)
• the struct should conform to an interface (c.f. gossip message interface already defined in shmsg)
• reject messages that are not parsable or have easily detectable invalid values

The libp2p validation interface should be able to do both parsing and validation at the same time. If not, revise.

• should support keys from G1
• needs support aggregate BLS signatures
• needs hashing for G1 (we can also implement this by ourselves)

Add subcommands to initialize database

• <root cmd> keyper initdb
• <root cmd> decryptor initdb

Should check that db doesn't exist yet and if so creates tables. For now just a placeholder table.

the identity should include the decryptor's BLS key

• Run the keypers
• Configure mocknode such that it sends messages encrypted with eon public key generated by keypers.
• Add eon public key to decryptor db
• Run decryptors

Check that messages are properly decrypted and signatures are produced.

(if possible, update readme in rs-play repo)

The repo should contain

• config files for keypers, decryptors, and mocknode(s)
• script(s) to setup the database(s)
• a readme that describes how you can run the test setup

example name: rolling-shutter-samples , rsspl, rllngshttrsmpls

Using gob if all else fails

The node currently generates a new random identity to be used for libp2p, so the address to be used by others to connect to it cannot be known in advance.

when the decryptor receives a key, they should check that it's valid. They can do so by comparing it against the eon public key and the epoch id. The eon public key is hardcoded as a constant somewhere -- this is a temporary solution. In the future, we might read it from Shuttermint or from a different blockchain

When the keyper receives a decryption trigger message for the first time, it should launch the epoch key generation process for the given epoch. In particular, as a result it should send an epoch key share message.

Right now they are just a single blob. This entails both message encoding and decryption.