Using EMA for BCH's new DA about difficulty-algorithms HOT 5 OPEN

Hmm, regarding the last graph, negative stolen seems fundamentally wrong to me ... by definition, a switch miner should only direct their hashrate at this chain when it is profitable, i.e., when the target they have to hit is strictly higher than some threshold. Since for TSA there is the distinction between actual target and the fake blockheader target, I wonder if your calculation might not be using the correct target?

from difficulty-algorithms.

I mentioned that indirectly in the 2nd to last sentence. Attacking for a loss does not make sense, so they will do something else like not attack. To clarify TSA, the attack trigger is if the N=80 baseline difficulty is below their threshold not the actual difficulty. Doing it this way implies they can't switch on and off quickly, losing less than about 10 seconds of mining time between two coin choices. If they can, then I'm missing is an important case that could harm the results enough to be more like the others. If large hashrate sources can switch so easily, they can wait until after T to get the lower-than avg difficulty, which will occur in < 1/e = 36% of the blocks. This will make post T solves very fast and drive up pre-T difficulty for everyone else. In studying optimal attacks for normal DAs, I noticed a very large miner could always get up to 1/3 of the blocks at zero cost without regard to the DA so 1/e =~ 1.3 of the blocks coming in at > T could indicate the rule still applies. The rule comes from the very large miner mining 2/3 of the blocks getting (effectively) 1/3 of the block at an average D cost, the other 1/3 at zero cost, and the remaining 1/3 that he's not mining would equal to 2x the average D.

You asked for data on the time-weighted avg difficulties for the above.
1374 SMA
1367 EMA 2-block delay
1355 EMA
1331 Mark RTT
1358 LWMA
1207 TSA

Constant HR: if there are no 6x attackers, avg D is 1000 in this simulation (at dedicated miners hashrate). The avg time-weighted D's for the regular DAs was 1000. The RTTs had 1.5% lower for Mark1 and 8% lower for TSA.

from difficulty-algorithms.

I've updated the beginning of this article to facilitate discussion.

from difficulty-algorithms.

There are a few practical things I wonder:

• Are the tighter timestamp rules going to affect miners adversely? I know timestamp rolling can theoretically be used for ASICboost, though it seems to be not used much in practice.
• The 144 / 72 numbers still sound arbitrary, like satoshi's 2016. There needs to be analysis of realistic price action with realistic switch/steady mining ratios to see what kind of numbers make sense. My gut feeling is that analysis would recommend a slower DAA (even a slow DAA can actually respond fairly quickly due to switch mining).
• I totally get the hesitance about putting clipping, however I am nervous that a DAA which permits huge difficulty drops would allow cheaply making low difficulty header chains to spam nodes (and cause out-of-memory): basically just one blocks' worth of PoW to drop difficulty to 1, then create a long string of difficulty-1 headers. I don't know if there is any non-rule-based (just optional) mitigation that can block these without also creating a sneaky chain split risk.

Finally, there are four rule changes proposed: 1) DAA, 2) Timestamp monotonicity, 3) FTL, 4) Clock. Even if they are simple changes and the final state is robust, the transition needs to be handled carefully and consider the interaction between all these changes happening around the same time.

from difficulty-algorithms.

I have not heard of any theory or observation that timestamps limits even as low as a few seconds have caused a problem.

Only 1 rule change needed. Keep current timestamp limits?
3 of the 4 rule changes can be avoided: Monotonicity can be done within the DA code (see bottom). FTL and clock can also be left alone. The potential damage from leaving FTL=7200 is a forwarded timestamp at that limit that would lower the next block's difficulty.
1/( 1+t/T/N-1/N) => 1/(1 + 7200/600/N - 1/N) = 13% easier difficulty.
The full 13% drop can be achieved only once per 2 hours, or spread out over it, benefiting the next block (not necessarily the "cheaters" difficulty) and penalizing others. The forced monotonicity (either by protocol of the DA) will make the next 10 or so honest solvetimes 1 second, increasing difficulty 1/(1-1/N) = 1.4% per block, bringing difficulty and avg solvetime back to the correct values. So the risk is mild. BTW if you clip the difficulty to not drop too much, say 5%, then the avg solvetime will be lower. I believe it means block emission will be 600 * (0.13-0.05) = 48 seconds behind, costing all of today's miners that much coin (8% of one reward).

N > 72 is probably a good idea
The 72 gives about the same std dev in difficulty under constant hashrate condition as the current SMA N=144 which is about 9%. See charts below. Given that miners are reacting to 5% changes with strong sensitivity I believe you are right that a longer window would be better. A problem is that the reduction in random variation is a function of SQRT(N) while the slowness to respond to HR changes is a function of N, so with higher N we get slower faster than we get stable.

Clipping drops in difficulty with limit on solvetime
Concerning the need for limited drops in difficulty, the method I've used in many coins is to limit the solvetime as used in the difficulty to 6xT which is very infrequent from random variation. This would allow a max drop in difficulty in a block of (6-1)/N = 7% for N=72. A longer window will reduce this. 5xT is possible, but not lower. This type of clipping can lead to larger-than-expected solvetimes if on-off mining triggers it a lot.

Currently, single block difficulty can rise or fall 10% due to long solvetimes coming into and out of the averaging window. With EMA, rises are limited to 1/N, so it gives miners more time to decide to leave, which reduces the chances that they all leave at once, preventing the long solve time if the DA reacts fast enough to drop as they leave. 1/72 = 1.4% and since they are sensitive to 5% changes, a larger N is again reasonable.

Code to prevent negative solvetimes

// We do not want to use MTP because of the 6-block delay, so we do this
// My source of the idea is T Harding who cites kyuupichan
// Catastrophic exploits result from attempting  if (st<1) { st=1;} 
// Attacker simply assigns old timestamp. Next honest stamp lower difficulty.
// Repeating this with 20% attacking HR if MTP(11)=6 quickly leads to a difficulty of 1.

previous_timestamp = timestamps[0];
for ( uint64_t i = 1; i <= 11; i++) {  // 11 is most recent block
      if (timestamps[i] > previous_timestamp  ) {   
            this_timestamp = timestamps[i];
      } else {  this_timestamp = previous_timestamp+1 ;   } // +1 sort of a safety measure
      solvetimes[i] = this_timestamp - previous_timestamp;
      previous_timestamp = this_timestamp;
}
// clip, if desired:
solvetime[11] = min[6*T, solvetime[11]);  
// EMA would use solvetime[11]

from difficulty-algorithms.