public inbox for bitcoindev@googlegroups.com
 help / color / mirror / Atom feed
From: Dave Hudson <dave@hashingit.com>
To: Bob McElrath <bob_bitcoin@mcelrath.org>
Cc: bitcoin-dev@lists.linuxfoundation.org
Subject: Re: [bitcoin-dev] Hardfork to fix difficulty drop algorithm
Date: Wed, 9 Mar 2016 23:24:15 +0000	[thread overview]
Message-ID: <DD5ED99B-6127-4D9F-9E39-B623234281BA@hashingit.com> (raw)
In-Reply-To: <20160309202135.GC4388@mcelrath.org>


> On 9 Mar 2016, at 20:21, Bob McElrath <bob_bitcoin@mcelrath.org> wrote:
> 
> Dave Hudson [dave@hashingit.com] wrote:
>> A damping-based design would seem like the obvious choice (I can think of a
>> few variations on a theme here, but most are found in the realms of control
>> theory somewhere).  The problem, though, is working working out a timeframe
>> over which to run the derivative calculations.
> 
> From a measurement theory perspective this is straightforward.  Each block is a
> measurement, and error propagation can be performed to derive an error on the
> derivatives.

Sure, but I think there are 2 problems:

1) My guess is that errors over anything but a long period are probably too large to be very useful.

2) We don't have a strong notion of time that is part of the consensus.  Sure, blocks have timestamps but they're very loosely controlled (can't be more than 2 hours ahead of what any validating node thinks the time might be).  Difficulty can't be calculated based on anything that's not part of the consensus data.

> The statistical theory of Bitcoin's block timing is known as a Poisson Point
> Process: https://en.wikipedia.org/wiki/Poisson_point_process or temporal point
> process.  If you google those plus "estimation" you'll find a metric shit-ton of
> literature on how to handle this.

Strictly it's a non-homogeneous Poisson Process, but I'm pretty familiar with the concept (Google threw one of my own blog posts back at me: http://hashingit.com/analysis/27-hash-rate-headaches, but I actually prefer this one: http://hashingit.com/analysis/30-finding-2016-blocks because most people seem to find it easier to visualize).

>> The problem is the measurement of the hashrate, which is pretty inaccurate at
>> best because even 2016 events isn't really enough (with a completely constant
>> hash rate running indefinitely we'd see difficulty swings of up to +/- 5% even
>> with the current algorithm).  In order to meaningfully react to a major loss
>> of hashing we'd still need to be considering a window of probably 2 weeks.
> 
> You don't want to assume it's constant in order to get a better measurement.
> The assumption is clearly false.  But, errors can be calculated, and retargeting
> can take errors into account, because no matter what we'll always be dealing
> with a finite sample.

Agreed, it's a thought experiment I ran in May 2014 (http://hashingit.com/analysis/28-reach-for-the-ear-defenders).  I found that many people's intuition is that there would be little or no difficulty changes in such a scenario, but the intuition isn't reliable.  Given a static hash rate the NHPP behaviour introduces a surprisingly large amount of noise (often much larger than any signal over a period of even weeks).  Any measurements in the order of even a few days has so much noise that it's practically unusable.  I just realized that unlike some of my other sims this one didn't make it to github; I'll fix that later this week.


Cheers,
Dave

  reply	other threads:[~2016-03-09 23:24 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-03-02 14:56 [bitcoin-dev] Hardfork to fix difficulty drop algorithm Luke Dashjr
2016-03-02 15:05 ` Pavel Janík
2016-03-02 15:14   ` Luke Dashjr
2016-03-02 15:24     ` Jérémie Dubois-Lacoste
     [not found]     ` <CAE-z3OUR8So2EM_EBeEerW-UPs0KY+whVB=jjFAHkW3xZPF2Hw@mail.gmail.com>
2016-03-02 15:54       ` Tier Nolan
2016-03-02 15:42 ` Luke Dashjr
2016-03-02 16:27   ` Paul Sztorc
2016-03-02 18:07     ` Tier Nolan
2016-03-02 19:01       ` Eric Voskuil
     [not found]         ` <56D74859.3090609@gmail.com>
2016-03-02 20:44           ` Eric Voskuil
2016-03-02 23:02         ` Peter Todd
2016-03-03  5:11           ` Dave Scotese
2016-03-03 10:14           ` Patrick Shirkey
2016-03-03 20:54           ` Eric Voskuil
2016-03-04 10:27             ` Tier Nolan
2016-03-02 15:48 ` Dave Hudson
2016-03-08 22:05   ` Bob McElrath
2016-03-09 18:30     ` Dave Hudson
2016-03-09 20:21       ` Bob McElrath
2016-03-09 23:24         ` Dave Hudson [this message]
2016-03-09 20:26       ` Paul Sztorc
2016-03-02 16:17 ` Bryan Bishop
2016-03-02 17:14 ` David A. Harding
2016-03-02 17:53   ` Gregory Maxwell
2016-03-02 19:34     ` David A. Harding
2016-03-03  1:06     ` Paul Sztorc
2016-03-09 17:58       ` Paul Sztorc
2016-03-02 18:20 ` Peter Todd
2016-03-03 18:27 ` Corey Haddad
2016-03-04  8:41   ` Henning Kopp
     [not found]     ` <CA+XQW1gfnXxxCod6cL=caGnEc66YOvaF6SJL=omUbMqwLNDP7g@mail.gmail.com>
2016-03-09 20:43       ` Paul Sztorc

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=DD5ED99B-6127-4D9F-9E39-B623234281BA@hashingit.com \
    --to=dave@hashingit.com \
    --cc=bitcoin-dev@lists.linuxfoundation.org \
    --cc=bob_bitcoin@mcelrath.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox