Misc questions about consensus-specs HOT 8 CLOSED

@mratsim Out of curiosity, what is a recovery byte?

from consensus-specs.

1. Should there be a check in the queuing logic that the number of validators is less than MAX_VALIDATOR_COUNT?

MAX_VALIDATOR_COUNT is not even used in the spec anywhere; we could just remove it.

2. Should slot and dynasty be set to int32 (as opposed to int64)? With 8 second slot durations 32 bits gives us 1000+ years. (Same for justified_streak, justified_slot, etc.)

Personally, I'm inclined to favor int64 over int32 for everything except data structures that get repeated a huge number of times; simplifies things that way. Though that's only my own preference; I'm ok with adding a millennium bug too.

3. Is pubkey being a int256 too small for BLS12-381?

Yep! Happy to make it an int384 when we switch over 😄

4. What happens if a shard block gets sufficient attestations to become a crosslink, but that shard block conflicts with an existing crosslink in the beacon chain?

This theoretically makes the entire beacon chain from that point invalid.

5. What calculation led to "~39.4%" in "the amount of time it takes for the quadratic leak to cut deposits of non-participating validators by ~39.4%"? Consider detailing with the constants.

0.606 ~= e ** -0.5. It's the easiest constant to work with in this context; basically, if we have a rule that deposits are cut by a factor of n/k^2 after n slots of inactivity leak, then after k slots deposits are

6. What calculation led to "~3.88%" in "~3.88% annual interest assuming 10 million participating ETH"? Consider details with the constants.

Calculating maximum rewards based on the given constants .

7. Should we allow deposits greater than DEPOSIT_SIZE? What about top-ups and penalties for being under the DEPOSIT_SIZE?

I brought up the possibility of making the deposit size variable earlier in our chat; I suppose it's a reasonable idea theoretically.

8. What is the maximum bias to the global clock that we are assuming honest validators to have?

< 16 seconds, though given other people want a smaller slot duration I guess it's <8 seconds 😄

9. Is it still possible that a dynasty transition can happen within a cycle?

Nope!

10. What happens if a validator deregisters and then wants to re-register?

They have to wait the ~3 months and then restart?

11. Do we need to define a SpecialObject to change the withdrawal_shard_id, withdrawal_address or randao_commitment?

IMO the withdrawal address and withdrawal shard should just be immutable; this way you can have a hot signing key and cold hodling key without risking the signing key being able to fully steal funds.

12. Should we deprecate Wei with 64-bit balances? Indeed, 64 bits of granualarity is enough for balances, and is consistent with a 64-bit EVM2.0?

Within the beacon chain, seems reasonable (use gwei as a unit?). Within shards, I'd say keep the use of bignums and wei for currency.

from consensus-specs.

I'm inclined to favor int64 over int32 for everything except data structures that get repeated a huge number of times; simplifies things that way

I'm happy with keeping things clean 👍. That suggests homogenising deposits_penalized_in_period, SpecialObject.type, ValidatorRecord.status, shard_id , withdrawal_shard and balance (with the suggestion of using Gwei as a unit) to int64.

This theoretically makes the entire beacon chain from that point invalid

Don't we expect conflicting crosslinks from time to time? Suppose for example a committee almost creates crosslink for a shard block but one of the signatures required to reach the committee threshold takes a bit of time to get into the beacon chain. Shortly after the following happen:

• The shard forks
• Another committee is sampled, and creates a crosslink for the fork
• The missing signature for the "old chain" makes it onchain

< 16 seconds, though given other people want a smaller slot duration I guess it's <8 seconds 😄

I think we should be precise about timing assumptions:

1. Are you suggesting that the subjective clocks of honest player be within UTC ± SLOT_DURATION (absolute bias around real time at most SLOT_DURATION) or within UTC ± SLOT_DURATION / 2 (relative bias between two honest players at most SLOT_DURATION)?
2. Does the above model include any networking latency? What are the networking latency assumptions?

They have to wait the ~3 months and then restart?

How do they restart? As currently specced the only way to register is via the one-deposit on the PoW chain. If deregistration happens near the launch of the beacon chain (prior to shards having state) it may take years for that validator to be able to restart. I guess my question is: do we want to add a SpecialObject for re-registration to avoid validators from being stuck for significantly longer than 3 months? If so, then we might as well have a unified SpecialObject for both registration and re-registration, and further simplify the registration contract on the PoW chain.

from consensus-specs.

Don't we expect conflicting crosslinks from time to time? Suppose for example a committee almost creates crosslink for a shard block but one of the signatures required to reach the committee threshold takes a bit of time to get into the beacon chain.

Ah I see. I was operating under the assumption that it's not possible to include a transaction for one committee by the time that it's possible to include a transaction for "the next committee" for that same shard, and particularly messages that contribute to committee N+1 must point to, and therefore be aware of, any blocks that could have included a transaction for committee N, so there's no room for honest misunderstandings from latency getting two conflicting committees into a beacon chain. I think that if we reduce the max committees per slot by a factor of at least 3 we get that invariant back; IMO that's the cleanest solution.

In any case, I really want to enforce the norm that if the beacon chain includes a crosslink from a given shard block, then that means the beacon chain attests to that shard block being part of the canonical history, and the ultimate validity of the beacon chain from that point on now is now conditional on the validity of the shard chain up until that point. Allowing conflicting crosslinks would make all of that impossible as the beacon chain would effectively be attesting to two conflicting beacon chains.

I think we should be precise about timing assumptions:

Technically, we want networking latency plus clock disparity between all honest nodes to be <= 1 slot, so if honest node A sends a message at subjective time T, honest node B will receive the message within the subjective time interval (T - 8, T + 8) (negatives possible due to clock disparity).

Though my simulations suggest things will be ok with clock disparity and/or network latency even going up to 3-4x slot duration or more, the chain just won't look as clean 😆

BTW I think we should consider whether or not a default client timing rule that adapts to some guess of what the majority of the network thinks the time is would make sense; will think more before making a concrete proposal.

How do they restart?

Using the shard-to-beacon-chain deposit procedure, which is TBD 😄

from consensus-specs.

BTW I think we should consider whether or not a default client timing rule that adapts to some guess of what the majority of the network thinks the time is would make sense; will think more before making a concrete proposal.

Prysm was planning on having clients sync with an NTP server: prysmaticlabs/prysm#511 (comment)

from consensus-specs.

3. Is pubkey being a int256 too small for BLS12-381?

Yep! Happy to make it an int384 when we switch over

The private key is 48 bytes (384-bit) but the public keys are 96 bytes, or 97 bytes if there is a recovery byte.

Edit: Ah, I was thinking about signatures not public keys, recovery bytes is on signatures.

from consensus-specs.

Taking from bitcoin's secp256k1:

/** Opaque data structured that holds a parsed ECDSA signature,
 *  supporting pubkey recovery.
 *
 *  The exact representation of data inside is implementation defined and not
 *  guaranteed to be portable between different platforms or versions. It is
 *  however guaranteed to be 65 bytes in size, and can be safely copied/moved.
 *  If you need to convert to a format suitable for storage or transmission, use
 *  the secp256k1_ecdsa_signature_serialize_* and
 *  secp256k1_ecdsa_signature_parse_* functions.
 *
 *  Furthermore, it is guaranteed that identical signatures (including their
 *  recoverability) will have identical representation, so they can be
 *  memcmp'ed.
 */
typedef struct {
    unsigned char data[65];
} secp256k1_ecdsa_recoverable_signature;

/** Parse a compact ECDSA signature (64 bytes + recovery id).
 *
 *  Returns: 1 when the signature could be parsed, 0 otherwise
 *  Args: ctx:     a secp256k1 context object
 *  Out:  sig:     a pointer to a signature object
 *  In:   input64: a pointer to a 64-byte compact signature
 *        recid:   the recovery id (0, 1, 2 or 3)
 */
SECP256K1_API int secp256k1_ecdsa_recoverable_signature_parse_compact(
    const secp256k1_context* ctx,
    secp256k1_ecdsa_recoverable_signature* sig,
    const unsigned char *input64,
    int recid
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3);

vs signature without recovery:

/** Opaque data structured that holds a parsed ECDSA signature.
 *
 *  The exact representation of data inside is implementation defined and not
 *  guaranteed to be portable between different platforms or versions. It is
 *  however guaranteed to be 64 bytes in size, and can be safely copied/moved.
 *  If you need to convert to a format suitable for storage, transmission, or
 *  comparison, use the secp256k1_ecdsa_signature_serialize_* and
 *  secp256k1_ecdsa_signature_parse_* functions.
 */
typedef struct {
    unsigned char data[64];
} secp256k1_ecdsa_signature;

...

/** Parse an ECDSA signature in compact (64 bytes) format.
 *
 *  Returns: 1 when the signature could be parsed, 0 otherwise.
 *  Args: ctx:      a secp256k1 context object
 *  Out:  sig:      a pointer to a signature object
 *  In:   input64:  a pointer to the 64-byte array to parse
 *
 *  The signature must consist of a 32-byte big endian R value, followed by a
 *  32-byte big endian S value. If R or S fall outside of [0..order-1], the
 *  encoding is invalid. R and S with value 0 are allowed in the encoding.
 *
 *  After the call, sig will always be initialized. If parsing failed or R or
 *  S are zero, the resulting sig value is guaranteed to fail validation for any
 *  message and public key.
 */
SECP256K1_API int secp256k1_ecdsa_signature_parse_compact(
    const secp256k1_context* ctx,
    secp256k1_ecdsa_signature* sig,
    const unsigned char *input64
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3);

In implementation, secp256k1 signature are represented with v, r and s and v is the recovery byte while r and s are 32 bytes for a total of 1 + 32 + 32 = 65 bytes.

Edit - here is additional explanation: ethereum/EIPs#155 (comment) by @jhoenicke

An ECDSA signature (r,s) is computed from a random nonce k: compute R = kG and set r = (R.x mod ORDER) and throw away R.y. To recover the public key one can use the formula 1/r(sR-hG), but that requires to recover R from the signature. If R.x < ORDER (which is almost always the case), then the missing part is R.y and there are two possibilities (one with low bit 0, one with low bit 1).
So one has to store the low bit of R.y in v.

low s normalization: If (r,s) is a valid signature then (r,(ORDER-s)) is also a valid signature, but it uses the "other R". Ethereum (and Bitcoin) require for some time that one always uses the smaller value for s, and then one has to flip R.y bit.

See also https://crypto.stackexchange.com/questions/18105/how-does-recovering-the-public-key-from-an-ecdsa-signature-work/18106

Edit 2: I was mixing public keys and signatures, the recovery byte is on signatures, public keys are 96 bytes. Signatures are 96 or 97 bytes.

from consensus-specs.

All issues addressed elsewhere 👍

from consensus-specs.