geo-protocol / observer Goto Github PK

The requests should not be sent.
The responses should be ignored.

Each GEO transaction contains Observers block number, after which claims and TSLs are not accepted any more.

It is very important to ensure some period of time, when claims are not accepted, but TSL are still accepted. Otherwise, it is possible for the malicious node to send claim right when the last block is processed and to make it impossible to send TSLs for each another node in the transaction.

   1                  2                    3
---+------------------+--------------------+-------------> T

1: Transaction created.
2: Claims are not accepted.
3: Claims and TSL are not accepted.

At this moment, block generation time = 1m and transaction dispute time = 20m;
Duration 2 - 1 must be 10m == 10 blocks;
Duration 3 -21 must be 10m == 10 blocks;

GEO Node must check observers chain at least 3 times during this 20 minutes, so the recommended timeout == 6m == 6 blocks.

At the the moment, Packet.hpp defines network packet header in the next way:

    typedef uint16_t PacketSize;
    typedef uint32_t ChannelIndex;

    // Note: one message may contains no more than 2**8 = 256 packets.
    // Max message size depends on max packet size:
    // for the public network (Internet transfer) the appropriate packet size is 508 bytes.
    // Packet header may contains 8 bytes. As the result, data segment of the packet may contains 500 byte.
    // The max message size = ~125kB.
    typedef uint8_t TotalPacketsCount;
    typedef TotalPacketsCount PacketIndex;

TotalPacketsCount must be increased to support larger messages, otherwise - transactions in the system would be heavily constrained in participants count.

Claims validation steps must include:

• Check each member is included only once.
• Check each signature is included only once.
• Check protocol version.
• Check integrity.

TSLs validation steps must include:

• Check there is claim with received TxID in chain.
• Check each member is included only once.
• Check each signature is included only once.
• Check each signature corresponds to related member (by MemberTxID).
• Check protocol version.
• Check integrity.

From time to time, pool instances synchronization seems to be broken. There are cases, when items are not propagated between observers. At this time, there is no additional info on what might cause the error.

Claims and TSL contains internal transaction IDs of nodes, which take part in operation.
It is important to ensure, that no internal IDs duplicates are present.

At this moment, blocks are generated right after consensus collected, and even if 100% observers are voting for the block, only consensus count is included into the block, and the rest are forced to sync. To prevent it, block producer must wait as long a possible and try to generate next block with as much signatures collected, as possible.

At the moment, the protocol is synchronous and doesn't supports multiple requests per connection, so it is enough to include under signature only typeID of the request and payload.

In case if protocol would be modified to support multiple requests per connection - requestID must be added under the signature too.

Roles

Og - observer that generates the block;
Or - observer that receives block candidates digests;

Description

At the moment, Or performs additional blockchain sync. in case if it receives BlockHash notification from the Og, and it's hash does not equal to the last block of the Or's chain. Og broadcasts BlockHash notification each time, when new block has been generated.

This kind of sync is needed for the cases, when Or has missed Signatures collection from the Og, and does not committed the last block, when the rest observers has been comited it. In this case, this sync. makes it possible for the Or to quickly download the last block form someone from the network and attach it to the chain.

Optimization

This synchronization might be performed when Or receives Block Digest from the Og and received digest's hash does not passes validation (does not corresponds to the last block hash). In this case Or should try to check validity of it's chain with the help of majority (perform synchronization).

This way makes it possible to simplify validation flow logic and optimize traffic usage.

Vulnerabilities

• Malicious Og might drain the Or with invalid Block Digests, so the Or would again and again perform sync. with other observers. To prevent this malicious behavior - only one sync. operation should be performed by the Or, and, in case if the last block of the Or was exactly the same as the majority has -- then Og should be marked as malicious node for current generation round, and no any sync. attempts must be performed.

In case if some pool instances was not sync well - it will try again in few seconds. In case if some observers is down - pool would try to sync with it every few seconds, and, as a result - waste huge amount of traffic.

It would be nice, if observers would try sync their pools with permanently increasing timeouts on failures. Also, it might be useful to use some kind of pong-notification which would be interpret by other observers as signal to start/restart pools sync.

GEO node checks it's transactions finality via asking observer about claims/tsls.
Each such request contains TxID, and observers must lookup this transactions in pools and in chain.

At the moment, chain iterates through all blocks and uses linear search.
It would be much better to have internal index TxID->claim / TxID->tsl.

func (chain *Chain) ContainsTx(TxID *transactions.TransactionUUID) uint64

At the moment, pool instances are stored in memory. It is OK for the Beta release, but it must be improved in RC.

Due to the chosen tokenomic model, observers are selected for some period of time (observers epoch, at the moment, the epoch is approximately equal to 1 week). epoch is measured in ethereum blocks, so each one observer's generated block must contains anchor to the ethereum chain (ethereum block number, during which this block was generated).

At the end of the epoch, next vote round should be done in ethereum, and the next observers list should be formed. This process is expected to be endless, so the observers list would be refreshed aproximately every epoch period.

In this model, there are 2 cases possible on epoch change:

Definitions

Current epoch: CE
Next epoch: NE.
Scenario: CE reaches the end. NE is several ethereum blocks from beginning.

Case 1: Majority of observers from NE is the same as from CE

In this case (if observers are not malicious) orphans in observers chain are not possible. Even if some part of observers chain has been changed, the majority set was not changed, so even in case if some block was generated after epoch change with observers set from CE - it is OK (it would pass validation in future, due to corresponding majority), and this block would not be dropped. In this scenario epoch change would be done softly (Soft Epoch Change).

Case 2. Majority of observers from NE significantly differs from majority of CE

In this case orphans in observers chain are possible. Due to the significant majority set changes, for some short period of time, there would be present 2 concurrent majorities sets, that are able to generate independent blockchains. This is due to Observers and GEO nodes notifications lag: there would be a period when some GEO nodes would already be informed about epoch change, and some would still know nothing. The same is true for observers.

In case if some GEO Nodes, that knows nothing about epoch change, would send their Claims to the majority from CE - this claims would be processed, so the GEO Nodes would receive OK response, but this claims would not be included in the final main chain, so there is a syncronisation failure possibility between GEO Nodes and Observers Chain. In worst case, GEO Nodes would commit the operation that would be dropped away from the observers chain as an orphan, but because GEO node thinks it is comitted -- it would never check it again and even might drop the TSL from it's internal storage.

This scenario is Hard Epoch Change.

The proposed solution

GEO Nodes must be able to predict the type of epoch change (Soft vs Hard) significantly before the change would happen. To do this, we should maintain 3 epochs in the same time:

• Current Epoch;
• The Next Epoch (that is known already, and no changes are possible);
• The Epoch Candidate (changeable observers list, that would freeze when it became Next Epoch).

In case if next change is Hard Epoch Change - GEO Nodes must ensure proper epoch change before any communications to the chain.

What should be done

• Claim period (period from transaction starting to the last block to which it might be included) must cover Hard Epoch Changes.

• GEO nodes must maintain epochs list.

• Observers must maintain epochs list.

In rare cases, when pool instances synchronization is impossible (for example, when majority of nodes are offline) - instances should be dropped to prevent infinite synchronization attempts and permanent memory usage.